Weekly Shaarli

All links of one week in a single page.

Week 09 (February 26, 2024)

Look, Ma, No Matrices!

This article talks about PGA (projective geometric algebra), an alternative approach to do geometric transformation that is not using matrix.

Machine Learning Is Not Like Your Brain - FULL SERIES - YouTube
thumbnail

This video specifically talked about the real difference in biological neurons and its artificial counterpart. I found the part is on how to represent information and training "artificial biological neuron networks" interesting. Notes (terms: neuron := biological neuron):

  • neurons accumulates the incoming charge (stateful), with charge leakage
  • three ways to encode information: frequency, timing, and parallelism
  • backprop breaks down on discrete signals
  • the only known mechanism for neuron weight update is "neurons that fires together wires together". but it's not very useful to developing a way to train it.
  • it's likely neurons interpret its input as binary.
  • the firing rate limit of neurons is only around 4 ms/spike (very slow)
  • directional hearing requires distinguishing 1/2 ms delay of signals. how is it possible given the maximum firing rate? answer: create a group of neurons each detecting different delay and ordering in input signal.
  • neuron components:
    • loopback neuron can store a bit of signal that can be set and reset. (like an SR latch)
    • mpsc (read-write) buffer
    • mechanism that repetitively reading refreshes the memory (like DRAM)
  • power efficiency. (12W for the brain) calculated result: neocortex only fires every two seconds (Wow!)