Abstract:A key driver of mammalian intelligence is the ability to represent incoming sensory information across multiple abstraction levels. For example, in the visual ventral stream, incoming signals are first represented as low-level edge filters and then transformed into high-level object representations. Similar hierarchical structures routinely emerge in artificial neural networks (ANNs) trained for object recognition tasks, suggesting that similar structures may underlie biological neural networks. However, the classical ANN training algorithm, backpropagation, is considered biologically implausible, and thus alternative biologically plausible training methods have been developed such as Equilibrium Propagation, Deep Feedback Control, Supervised Predictive Coding, and Dendritic Error Backpropagation. Several of those models propose that local errors are calculated for each neuron by comparing apical and somatic activities. Notwithstanding, from a neuroscience perspective, it is not clear how a neuron could compare compartmental signals. Here, we propose a solution to this problem in that we let the apical feedback signal change the postsynaptic firing rate and combine this with a differential Hebbian update, a rate-based version of classical spiking time-dependent plasticity (STDP). We prove that weight updates of this form minimize two alternative loss functions that we prove to be equivalent to the error-based losses used in machine learning: the inference latency and the amount of top-down feedback necessary. Moreover, we show that the use of differential Hebbian updates works similarly well in other feedback-based deep learning frameworks such as Predictive Coding or Equilibrium Propagation. Finally, our work removes a key requirement of biologically plausible models for deep learning and proposes a learning mechanism that would explain how temporal Hebbian learning rules can implement supervised hierarchical learning.

Hierarchical Learning Induces Two Simultaneous, But Separable, Prediction Errors in Human Basal Ganglia

Dopaminergic prediction errors in the ventral tegmental area reflect a multithreaded predictive model

Electrophysiological Signatures of Hierarchical Learning

Parallel Contributions of Distinct Human Memory Systems During Probabilistic Learning.

Hierarchy of prediction errors shapes the learning of context-dependent sensory representations

The many worlds hypothesis of dopamine prediction error: implications of a parallel circuit architecture in the basal ganglia

Dopamine neuron ensembles signal the content of sensory prediction errors

A computational hierarchy in human cortex

Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework

Learning cortical hierarchies with temporal Hebbian updates

Beyond Reward Prediction Errors: Human Striatum Updates Rule Values During Learning

The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning

Mechanisms of Hierarchical Reinforcement Learning in Corticostriatal Circuits 1: Computational Analysis

Dopaminergic responses to identity prediction errors depend differently on the orbitofrontal cortex and hippocampus

Subsecond fluctuations in extracellular dopamine encode reward and punishment prediction errors in humans

Dopamine neurons report an error in the temporal prediction of reward during learning

Pre-existing visual responses in a projection-defined dopamine population explain individual learning trajectories

Dopamine, Prediction Error and Beyond

Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward