Abstract:Over the past several decades, backpropagation (BP) has played a critical role in the advancement of machine learning and remains a core method in numerous computational applications. It is also utilized extensively in comparative studies of biological and artificial neural network representations. Despite its widespread use, the implementation of BP in the brain remains elusive, and its biological plausibility is often questioned due to inherent issues such as the need for symmetry of weights between forward and backward connections, and the requirement of distinct forward and backward phases of computation. Here, we introduce a novel neuroplasticity rule that offers a potential mechanism for implementing BP in the brain. Similar in general form to the classical Hebbian rule, this rule is based on the core principles of maintaining the balance of excitatory and inhibitory inputs as well as on retrograde signaling, and operates over three progressively slower timescales: neural firing, retrograde signaling, and neural plasticity. We hypothesize that each neuron possesses an internal state, termed credit, in addition to its firing rate. After achieving equilibrium in firing rates, neurons receive credits based on their contribution to the E-I balance of postsynaptic neurons through retrograde signaling. As the network's credit distribution stabilizes, connections from those presynaptic neurons are strengthened that significantly contribute to the balance of postsynaptic neurons. We demonstrate mathematically that our learning rule precisely replicates BP in layered neural networks without any approximations. Simulations on artificial neural networks reveal that this rule induces varying community structures in networks, depending on the learning rate. This simple theoretical framework presents a biologically plausible implementation of BP, with testable assumptions and predictions that may be evaluated through biological experiments.

Balance Between Performance and Robustness of Recurrent Neural Networks Brought by Brain-Inspired Constraints on Initial Structure

Biologically-Informed Excitatory and Inhibitory Balance for Robust Spiking Neural Network Training

Biologically Inspired Structure Learning with Reverse Knowledge Distillation for Spiking Neural Networks

Balanced Excitation and Inhibition are Required for High-Capacity, Noise-Robust Neuronal Selectivity

The brain as an efficient and robust adaptive learner

Exploring weight initialization, diversity of solutions, and degradation in recurrent neural networks trained for temporal and decision-making tasks

Recurrent neural circuits overcome partial inactivation by compensation and re-learning

A Study of Biologically Plausible Neural Network: The Role and Interactions of Brain-Inspired Mechanisms in Continual Learning

Balancing Selectivity and Generality in Object Recognition through Structured Interconnectivity.

Adaptive Synaptic Scaling in Spiking Networks for Continual Learning and Enhanced Robustness

Tuning Convolutional Spiking Neural Network with Biologically-plausible Reward Propagation

Brain topology improved spiking neural network for efficient reinforcement learning of continuous control

Growing Brains: Co-emergence of Anatomical and Functional Modularity in Recurrent Neural Networks

Normative framework for deriving neural networks with multi-compartmental neurons and non-Hebbian plasticity

Oja's plasticity rule overcomes several challenges of training neural networks under biological constraints

Structured flexibility in recurrent neural networks via neuromodulation

Control of synaptic plasticity in neural networks

Rich dynamics caused by known biological brain network features resulting in stateful networks

Optimal modularity and memory capacity of neural reservoirs

Composing Recurrent Spiking Neural Networks using Locally-Recurrent Motifs and Risk-Mitigating Architectural Optimization

Contribute to balance, wire in accordance: Emergence of backpropagation from a simple, bio-plausible neuroplasticity rule