Abstract:Even in the absence of sensory stimulation the brain is spontaneously active. This background "noise" seems to be the dominant cause of the notoriously high trial-to-trial variability of neural recordings. Recent experimental observations have extended our knowledge of trial-to-trial variability and spontaneous activity in several directions: 1. Trial-to-trial variability systematically decreases following the onset of a sensory stimulus or the start of a motor act. 2. Spontaneous activity states in sensory cortex outline the region of evoked sensory responses. 3. Across development, spontaneous activity aligns itself with typical evoked activity patterns. 4. The spontaneous brain activity prior to the presentation of an ambiguous stimulus predicts how the stimulus will be interpreted. At present it is unclear how these observations relate to each other and how they arise in cortical circuits. Here we demonstrate that all of these phenomena can be accounted for by a deterministic self-organizing recurrent neural network model (SORN), which learns a predictive model of its sensory environment. The SORN comprises recurrently coupled populations of excitatory and inhibitory threshold units and learns via a combination of spike-timing dependent plasticity (STDP) and homeostatic plasticity mechanisms. Similar to balanced network architectures, units in the network show irregular activity and variable responses to inputs. Additionally, however, the SORN exhibits sequence learning abilities matching recent findings from visual cortex and the network's spontaneous activity reproduces the experimental findings mentioned above. Intriguingly, the network's behaviour is reminiscent of sampling-based probabilistic inference, suggesting that correlates of sampling-based inference can develop from the interaction of STDP and homeostasis in deterministic networks. We conclude that key observations on spontaneous brain activity and the variability of neural responses can be accounted for by a simple deterministic recurrent neural network which learns a predictive model of its sensory environment via a combination of generic neural plasticity mechanisms.

Representational drift as a result of implicit regularization

Representational drift as a result of implicit regularization

Representational drift as the consequence of ongoing memory storage

Stochastic Gradient Descent-Induced Drift of Representation in a Two-Layer Neural Network

Drift of neural ensembles driven by slow fluctuations of intrinsic excitability

Stability through plasticity: Finding robust memories through representational drift

Deep learning-based location decoding reveals that across-day representational drift is better predicted by rewarded experience than time

Heavy-tailed statistics of cortical representational drift are advantageous for stabilised downstream readouts

Representational Drift and Learning-Induced Stabilization in the Olfactory Cortex

Quantifying stimulus-relevant representational drift using cross-modality contrastive learning

One Step Back, Two Steps Forward: Interference and Learning in Recurrent Neural Networks

Sensory experience steers representational drift in mouse visual cortex

Incremental Self-Organization of Spatio-Temporal Spike Pattern Detection

Selective consistency of recurrent neural networks induced by plasticity as a mechanism of unsupervised perceptual learning

A biologically inspired architecture with switching units can learn to generalize across backgrounds

The cost of behavioral flexibility: reversal learning driven by a spiking neural network

A Study of Biologically Plausible Neural Network: The Role and Interactions of Brain-Inspired Mechanisms in Continual Learning

Where’s the Noise? Key Features of Spontaneous Activity and Neural Variability Arise through Learning in a Deterministic Network

Neural variability compresses with increasing belief precision during Bayesian inference

Continual learning under domain transfer with sparse synaptic bursting