Abstract:Neurons in the brain have rich and adaptive input-output properties. Features such as heterogeneous f-I curves and spike frequency adaptation are known to place single neurons in optimal coding regimes when facing changing stimuli. Yet, it is still unclear how brain circuits exploit single-neuron flexibility, and how network-level requirements may have shaped such cellular function. To answer this question, a multi-scaled approach is needed where the computations of single neurons and neural circuits must be considered as a complete system. In this work, we use artificial neural networks to systematically investigate single-neuron input-output adaptive mechanisms, optimized in an end-to-end fashion. Throughout the optimization process, each neuron has the liberty to modify its nonlinear activation function parametrized to mimic f-I curves of biological neurons, either by learning an individual static function or via a learned and shared adaptation mechanism to modify activation functions in real-time during a task. We find that such adaptive networks show much-improved robustness to noise and changes in input statistics. Using tools from dynamical systems theory, we analyze the role of these emergent single-neuron properties and argue that neural diversity and adaptation play an active regularization role, enabling neural circuits to optimally propagate information across time. Finally, we outline similarities between these optimized solutions and known coding strategies found in biological neurons, such as gain scaling and fractional order differentiation/integration. Evolution has shaped neural circuits in the brain to support complex tasks and behavior. In doing so, single neurons have developed intriguing coding properties such as heterogeneous and adaptive nonlinear responses, and mechanisms at network levels have emerged to efficiently propagate and process information. Discerning the impact of these distinct scales in shaping biophysical mechanisms is a difficult proposition. Are circuits wired the way they are because of single-neuron properties? Are single neurons behaving the way they do because of circuit requirements? To tackle this question, we turn to neural network models that we optimize with deep learning techniques. Crucially, we endow single neurons in these models with parameters that govern their input-output properties, as well as adaptation dynamics. These mechanisms are then optimized end-to-end together with network parameters to perform various perception-like tasks. While this process could have produced any novel solution, we find that what naturally emerges from optimization bears noteworthy similarity to biophysical mechanisms found in neurons in the brain, such as spike-frequency adaptation and gain scaling. We perform an in-depth analysis of the optimization process that uncovered these strategies and formulate hypotheses relating joint evolutionary pressures of single-neuron properties and circuit-level requirements in the brain.

The Impact of Spike-Frequency Adaptation on Balanced Network Dynamics

Effects of Spike Frequency Adaptation on Synchronization Transitions in Electrically Coupled Neuronal Networks with Scale-Free Connectivity

Spike-frequency Adaptation of a Two-Compartment Neuron Modulated by Extracellular Electric Fields

Impact of adaptation currents on synchronization of coupled exponential integrate-and-fire neurons

A Spike-Based Model of Neuronal Intrinsic Plasticity.

Spike Initiating Dynamics of the Neuron with Different Adaptation Mechanisms to Extracellular Electric Fields

Dynamics of the Exponential Integrate-and-fire Model with Slow Currents and Adaptation

Population spiking and bursting in next-generation neural masses with spike-frequency adaptation

Balanced Networks of Spiking Neurons with Spatially Dependent Recurrent Connections

Contributions of Adaptation Currents to Dynamic Spike Threshold on Slow Timescales: Biophysical Insights from Conductance-Based Models

The tuning of tuning: How adaptation influences single cell information transfer

ADAPTATION-DEPENDENT SYNCHRONIZATION TRANSITIONS AND BURST GENERATIONS IN ELECTRICALLY COUPLED NEURAL NETWORKS

SPIKE-FREQUENCY ADAPTATION IN THE MODIFIED LEAKY INTEGRATE-AND-FIRE MODEL UNDER EXTERNAL ELECTRIC FIELD

A robust balancing mechanism for spiking neural networks

Spike frequency adaptation in primate lateral prefrontal cortex neurons results from interplay between intrinsic properties and circuit dynamics

Adaptive Synaptic Scaling in Spiking Networks for Continual Learning and Enhanced Robustness

Adaptation controls synchrony and cluster states of coupled threshold-model neurons

Synaptic Plasticity in Correlated Balanced Networks

Neural networks with optimized single-neuron adaptation uncover biologically plausible regularization

Dynamic Information Encoding With Dynamic Synapses in Neural Adaptation.

Synchronization and Oscillation Behaviors of Excitatory and Inhibitory Populations with Spike-Timing-dependent Plasticity