1690 shaares
869 private links
869 private links
This video specifically talked about the real difference in biological neurons and its artificial counterpart. I found the part is on how to represent information and training "artificial biological neuron networks" interesting. Notes (terms: neuron := biological neuron):
- neurons accumulates the incoming charge (stateful), with charge leakage
- three ways to encode information: frequency, timing, and parallelism
- backprop breaks down on discrete signals
- the only known mechanism for neuron weight update is "neurons that fires together wires together". but it's not very useful to developing a way to train it.
- it's likely neurons interpret its input as binary.
- the firing rate limit of neurons is only around 4 ms/spike (very slow)
- directional hearing requires distinguishing 1/2 ms delay of signals. how is it possible given the maximum firing rate? answer: create a group of neurons each detecting different delay and ordering in input signal.
- neuron components:
- loopback neuron can store a bit of signal that can be set and reset. (like an SR latch)
- mpsc (read-write) buffer
- mechanism that repetitively reading refreshes the memory (like DRAM)
- power efficiency. (12W for the brain) calculated result: neocortex only fires every two seconds (Wow!)