Daily Shaarli

All links of one day in a single page.

February 29, 2024

Machine Learning Is Not Like Your Brain - FULL SERIES - YouTube
thumbnail

This video specifically talked about the real difference in biological neurons and its artificial counterpart. I found the part is on how to represent information and training "artificial biological neuron networks" interesting. Notes (terms: neuron := biological neuron):

  • neurons accumulates the incoming charge (stateful), with charge leakage
  • three ways to encode information: frequency, timing, and parallelism
  • backprop breaks down on discrete signals
  • the only known mechanism for neuron weight update is "neurons that fires together wires together". but it's not very useful to developing a way to train it.
  • it's likely neurons interpret its input as binary.
  • the firing rate limit of neurons is only around 4 ms/spike (very slow)
  • directional hearing requires distinguishing 1/2 ms delay of signals. how is it possible given the maximum firing rate? answer: create a group of neurons each detecting different delay and ordering in input signal.
  • neuron components:
    • loopback neuron can store a bit of signal that can be set and reset. (like an SR latch)
    • mpsc (read-write) buffer
    • mechanism that repetitively reading refreshes the memory (like DRAM)
  • power efficiency. (12W for the brain) calculated result: neocortex only fires every two seconds (Wow!)