Abstract:A key driver of mammalian intelligence is the ability to represent incoming sensory information across multiple abstraction levels. For example, in the visual ventral stream, incoming signals are first represented as low-level edge filters and then transformed into high-level object representations. Similar hierarchical structures routinely emerge in artificial neural networks (ANNs) trained for object recognition tasks, suggesting that similar structures may underlie biological neural networks. However, the classical ANN training algorithm, backpropagation, is considered biologically implausible, and thus alternative biologically plausible training methods have been developed such as Equilibrium Propagation, Deep Feedback Control, Supervised Predictive Coding, and Dendritic Error Backpropagation. Several of those models propose that local errors are calculated for each neuron by comparing apical and somatic activities. Notwithstanding, from a neuroscience perspective, it is not clear how a neuron could compare compartmental signals. Here, we propose a solution to this problem in that we let the apical feedback signal change the postsynaptic firing rate and combine this with a differential Hebbian update, a rate-based version of classical spiking time-dependent plasticity (STDP). We prove that weight updates of this form minimize two alternative loss functions that we prove to be equivalent to the error-based losses used in machine learning: the inference latency and the amount of top-down feedback necessary. Moreover, we show that the use of differential Hebbian updates works similarly well in other feedback-based deep learning frameworks such as Predictive Coding or Equilibrium Propagation. Finally, our work removes a key requirement of biologically plausible models for deep learning and proposes a learning mechanism that would explain how temporal Hebbian learning rules can implement supervised hierarchical learning.

NMDA-driven dendritic modulation enables multitask representation learning in hierarchical sensory processing pathways

Learning to combine top-down context and feed-forward representations under ambiguity with apical and basal dendrites

An Overlooked Role of Context-Sensitive Dendrites

Synergistic pathways of modulation enable robust task packing within neural dynamics

Rapid context inference in a thalamocortical model using recurrent neural networks

Neuromodulators generate multiple context-relevant behaviors in a recurrent neural network by shifting activity flows in hyperchannels

Learning cortical hierarchies with temporal Hebbian updates

Beyond spiking networks: The computational advantages of dendritic amplification and input segregation

Unleashing the Potential of Spiking Neural Networks for Sequential Modeling with Contextual Embedding.

Context association in pyramidal neurons through local synaptic plasticity in apical dendrites

Modelling continual learning in humans with Hebbian context gating and exponentially decaying task signals

Hierarchy of prediction errors shapes the learning of context-dependent sensory representations

A Geometric Framework for Understanding Dynamic Information Integration in Context-Dependent Computation

Local dendritic balance enables learning of efficient representations in networks of spiking neurons

Spike-driven Multi-Scale Learning with Hybrid Mechanisms of Spiking Dendrites.

Dendritic error backpropagation in deep cortical microcircuits

Learning reshapes the hippocampal representation hierarchy

Unsigned temporal difference errors in cortical L5 dendrites during learning

Temporal dendritic heterogeneity incorporated with spiking neural networks for learning multi-timescale dynamics

Dentate gyrus ensembles gate context-dependent neural states and memory retrieval