Abstract:A key driver of mammalian intelligence is the ability to represent incoming sensory information across multiple abstraction levels. For example, in the visual ventral stream, incoming signals are first represented as low-level edge filters and then transformed into high-level object representations. Similar hierarchical structures routinely emerge in artificial neural networks (ANNs) trained for object recognition tasks, suggesting that similar structures may underlie biological neural networks. However, the classical ANN training algorithm, backpropagation, is considered biologically implausible, and thus alternative biologically plausible training methods have been developed such as Equilibrium Propagation, Deep Feedback Control, Supervised Predictive Coding, and Dendritic Error Backpropagation. Several of those models propose that local errors are calculated for each neuron by comparing apical and somatic activities. Notwithstanding, from a neuroscience perspective, it is not clear how a neuron could compare compartmental signals. Here, we propose a solution to this problem in that we let the apical feedback signal change the postsynaptic firing rate and combine this with a differential Hebbian update, a rate-based version of classical spiking time-dependent plasticity (STDP). We prove that weight updates of this form minimize two alternative loss functions that we prove to be equivalent to the error-based losses used in machine learning: the inference latency and the amount of top-down feedback necessary. Moreover, we show that the use of differential Hebbian updates works similarly well in other feedback-based deep learning frameworks such as Predictive Coding or Equilibrium Propagation. Finally, our work removes a key requirement of biologically plausible models for deep learning and proposes a learning mechanism that would explain how temporal Hebbian learning rules can implement supervised hierarchical learning.

Anti-Hebbian learning in topologically constrained linear networks: a tutorial

Modeling and Contractivity of Neural-Synaptic Networks with Hebbian Learning

Dynamics Analysis and Analog Associative Memory of Networks with LT Neurons.

Learning Connectivity of Neural Networks from a Topological Perspective

Strong anti-Hebbian plasticity alters the convexity of network attractor landscapes

Neuromimetic Dynamic Networks with Hebbian Learning

Hebbian Crosstalk Prevents Nonlinear Unsupervised Learning

Nonlinear Hebbian learning as a unifying principle in receptive field formation

Hebbian Learning from First Principles

Unsupervised learning by a nonlinear network with Hebbian excitatory and anti-Hebbian inhibitory neurons

A Mathematical Analysis of the Effects of Hebbian Learning Rules on the Dynamics and Structure of Discrete-Time Random Recurrent Neural Networks

Neuron-centric Hebbian Learning

Hebbian And Anti-Hebbian Learning For Independent Component Analysis

Analysis of Cyclic Dynamics for Networks of Linear Threshold Neurons.

A simple normative network approximates local non-Hebbian learning in the cortex

Why do similarity matching objectives lead to Hebbian/anti-Hebbian networks?

Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks

A Rigorous Analysis of Linsker's Hebbian Learning Network

Layerwise Hebbian/anti-Hebbian (HaH) Learning In Deep Networks: A Neuro-inspired Approach To Robustness

A spectral approach to Hebbian-like neural networks

Learning cortical hierarchies with temporal Hebbian updates