Abstract:Individual neurons in the brain have complex intrinsic dynamics that are highly diverse. We hypothesize that the complex dynamics produced by networks of complex and heterogeneous neurons may contribute to the brain's ability to process and respond to temporally complex data. To study the role of complex and heterogeneous neuronal dynamics in network computation, we develop a rate-based neuronal model, the generalized-leaky-integrate-and-fire-rate (GLIFR) model, which is a rate equivalent of the generalized-leaky-integrate-and-fire model. The GLIFR model has multiple dynamical mechanisms, which add to the complexity of its activity while maintaining differentiability. We focus on the role of after-spike currents, currents induced or modulated by neuronal spikes, in producing rich temporal dynamics. We use machine learning techniques to learn both synaptic weights and parameters underlying intrinsic dynamics to solve temporal tasks. The GLIFR model allows the use of standard gradient descent techniques rather than surrogate gradient descent, which has been used in spiking neural networks. After establishing the ability to optimize parameters using gradient descent in single neurons, we ask how networks of GLIFR neurons learn and perform on temporally challenging tasks, such as sequential MNIST. We find that these networks learn diverse parameters, which gives rise to diversity in neuronal dynamics, as demonstrated by clustering of neuronal parameters. GLIFR networks have mixed performance when compared to vanilla recurrent neural networks, with higher performance in pixel-by-pixel MNIST but lower in line-by-line MNIST. However, they appear to be more robust to random silencing. We find that the ability to learn heterogeneity and the presence of after-spike currents contribute to these gains in performance. Our work demonstrates both the computational robustness of neuronal complexity and diversity in networks and a feasible method of training such models using exact gradients.

Effect of heterogeneity for the self-organization of a neural network

Self-Organization Of A Neural Network With Heterogeneous Neurons Enhances Coherence And Stochastic Resonance

Effect of noises on two-layer hodgkin-huxley neuronal network

Firing Rate Dynamics in Recurrent Spiking Neural Networks with Intrinsic and Network Heterogeneity

Variable synaptic strengths controls the firing rate distribution in feedforward neural networks

Dive into the Power of Neuronal Heterogeneity

Effect of Synaptic Heterogeneity on Neuronal Coordination

Stochastic and Coherence Resonance in Feed-Forward-Loop Neuronal Network Motifs

Stochastic Resonance in Feedforward-Loop Neuronal Network Motifs in Astrocyte Field

Heterogeneous Neuronal and Synaptic Dynamics for Spike-Efficient Unsupervised Learning: Theory and Design Principles

Neuronal avalanche dynamics regulated by spike-timing-dependent plasticity under different topologies and heterogeneities

Optimal responsiveness and collective oscillations emerging from the heterogeneity of inhibitory neurons

Collective Behaviors of Neural Network Regulated by the Spatially Distributed Stimuli

Dynamics and synchronization in a memristor-coupled discrete heterogeneous neuron network considering noise

Heterogeneity of Synaptic Input Connectivity Regulates Spike-Based Neuronal Avalanches.

Neuronal heterogeneity modulates phase-synchronization between unidirectionally coupled populations with excitation-inhibition balance

A Coarse-Grained Framework for Spiking Neuronal Networks: Between Homogeneity and Synchrony

Robust Computation with Intrinsic Heterogeneity

Heterogeneity in Neuronal Dynamics Is Learned by Gradient Descent for Temporal Processing Tasks

Self-organization of microcircuits in networks of neurons with plastic synapses

Drifting states and synchronization induced chaos in autonomous networks of excitable neurons