Abstract:Intrinsic within-type neuronal heterogeneity is a ubiquitous feature of biological systems, with well-documented computational advantages. Recent works in machine learning have incorporated such diversities by optimizing neuronal parameters alongside synaptic connections and demonstrated state-of-the-art performance across common benchmarks. However, this performance gain comes at the cost of significantly higher computational costs, imposed by a larger parameter space. Furthermore, it is unclear how the neuronal parameters, constrained by the biophysics of their surroundings, are globally orchestrated to minimize top-down errors. To address these challenges, we postulate that neurons are intrinsically diverse, and investigate the computational capabilities of such heterogeneous neuronal parameters. Our results show that intrinsic heterogeneity, viewed as a fixed quenched disorder, often substantially improves performance across hundreds of temporal tasks. Notably, smaller but heterogeneous networks outperform larger homogeneous networks, despite consuming less data. We elucidate the underlying mechanisms driving this performance boost and illustrate its applicability to both rate and spiking dynamics. Moreover, our findings demonstrate that heterogeneous networks are highly resilient to severe alterations in their recurrent synaptic hyperparameters, and even recurrent connections removal does not compromise performance. The remarkable effectiveness of heterogeneous networks with small sizes and relaxed connectivity is particularly relevant for the neuromorphic community, which faces challenges due to device-to-device variability. Furthermore, understanding the mechanism of robust computation with heterogeneity also benefits neuroscientists and machine learners.

Diversity-induced trivialization and resilience of neural dynamics

Neural Heterogeneity Enhances Reliable Neural information Processing: Local Sensitivity and Globally Input-slaved Transient Dynamics

Diversity and time delays induce resonance in a modular neuronal network.

From asynchronous states to Griffiths phases and back: structural heterogeneity and homeostasis in excitatory-inhibitory networks

Amplified Signal Response by Neuronal Diversity on Complex Networks

Optimal responsiveness and collective oscillations emerging from the heterogeneity of inhibitory neurons

Neuronal diversity can improve machine learning for physics and beyond

Firing Rate Dynamics in Recurrent Spiking Neural Networks with Intrinsic and Network Heterogeneity

Enhancement of Neuronal Coherence by Diversity in Coupled Rulkov-Map Models

Robust Computation with Intrinsic Heterogeneity

IMPACT OF DIVERSITY AND DELAYS ON THE RESONANCE DYNAMICS OF NEURONAL NETWORKS

Dive into the Power of Neuronal Heterogeneity

Diversity Begets Stability in an Evolving Network

Heterogeneity of Synaptic Input Connectivity Regulates Spike-Based Neuronal Avalanches.

A biophysical perspective on the resilience of neuronal excitability across timescales

Resilience for stochastic systems interacting via a quasi-degenerate network

No neuron is an island: Homeostatic plasticity and over-constraint in a neural circuit

Variable synaptic strengths controls the firing rate distribution in feedforward neural networks

Self-Organization Of A Neural Network With Heterogeneous Neurons Enhances Coherence And Stochastic Resonance

Effect of Synaptic Heterogeneity on Neuronal Coordination

Dynamical mean-field theory for a highly heterogeneous neural population