Abstract:A key driver of mammalian intelligence is the ability to represent incoming sensory information across multiple abstraction levels. For example, in the visual ventral stream, incoming signals are first represented as low-level edge filters and then transformed into high-level object representations. Similar hierarchical structures routinely emerge in artificial neural networks (ANNs) trained for object recognition tasks, suggesting that similar structures may underlie biological neural networks. However, the classical ANN training algorithm, backpropagation, is considered biologically implausible, and thus alternative biologically plausible training methods have been developed such as Equilibrium Propagation, Deep Feedback Control, Supervised Predictive Coding, and Dendritic Error Backpropagation. Several of those models propose that local errors are calculated for each neuron by comparing apical and somatic activities. Notwithstanding, from a neuroscience perspective, it is not clear how a neuron could compare compartmental signals. Here, we propose a solution to this problem in that we let the apical feedback signal change the postsynaptic firing rate and combine this with a differential Hebbian update, a rate-based version of classical spiking time-dependent plasticity (STDP). We prove that weight updates of this form minimize two alternative loss functions that we prove to be equivalent to the error-based losses used in machine learning: the inference latency and the amount of top-down feedback necessary. Moreover, we show that the use of differential Hebbian updates works similarly well in other feedback-based deep learning frameworks such as Predictive Coding or Equilibrium Propagation. Finally, our work removes a key requirement of biologically plausible models for deep learning and proposes a learning mechanism that would explain how temporal Hebbian learning rules can implement supervised hierarchical learning.

Beyond spiking networks: The computational advantages of dendritic amplification and input segregation

Burst-dependent plasticity and dendritic amplification support target-based learning and hierarchical imitation learning

Toward Efficient Processing and Learning with Spikes: New Approaches for Multispike Learning

Spike-driven Multi-Scale Learning with Hybrid Mechanisms of Spiking Dendrites.

Efficient Spike-Driven Learning With Dendritic Event-Based Processing

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Dendritic error backpropagation in deep cortical microcircuits

NADOL: Neuromorphic Architecture for Spike-Driven Online Learning by Dendrites

Two-compartment neuronal spiking model expressing brain-state specific apical-amplification, -isolation and -drive regimes

Investigation on the Effect of Spiking Neuron Properties with Dendrite Compartments in Brain-inspired Context-dependent Learning

Enhancing learning in spiking neural networks through neuronal heterogeneity and neuromodulatory signaling

A Spiking Network for Inference of Relations Trained with Neuromorphic Backpropagation

A solution to the learning dilemma for recurrent networks of spiking neurons

The brain as an efficient and robust adaptive learner

Leveraging dendritic properties to advance machine learning and neuro-inspired computing

Optimality of short-term synaptic plasticity in modelling certain dynamic environments

Deep Learning Depends on Segregated Dendritic Compartments and Spike Frequency Adaptation

Learning cortical hierarchies with temporal Hebbian updates

Dendritic-Inspired Processing Enables Bio-Plausible STDP in Compound Binary Synapses

Active Dendrites Enable Efficient Continual Learning in Time-To-First-Spike Neural Networks