Abstract:Previous works investigating representational drift from sensory to central nervous systems converged to show that neural coding, especially at the population level, readily overcomes these session-to-session fluctuations. However, representational drift in the primary visual cortex is more prominent when presenting naturalistic stimuli than artificial stimuli. Animals continuously navigate natural environments during the evolutionary timescale. Why did evolution not get rid of representational drift if it was just an inconvenience? Here, we investigate how representational drift simultaneously influences the encoding of multiple behaviorally relevant features in a natural movie stimulus. Because natural environments contain multiple interacting spatio-temporal features, previous works only provided incomplete understanding of representational drift because of such simplification. Here, we use cross modality contrastive learning to learn an embedding of neural activity that retains only those relevant components of the natural movie stimulus. We also observe that our learned embedding is near-optimal in decoding a whole suite of natural features (scene, optic flow, complex spatio-temporal features, and time) and generalizable to decode those features from single-trial or novel hold-out data. Using this embedding as a surrogate model, we observe that representational drift perturbs the local geometry of the embedding, and this results in various changes in performance when we decode from a different session (90 min later) even at the population level. Our work further suggests that a separate compensation mechanism may be necessary for the optic flow features, as their autocorrelation scale is shorter than the minimum time needed to discriminate scene texture features. Thus, representational drift may encourage neural processing flexibility rather than be a mere nuisance.

Sensory experience steers representational drift in mouse visual cortex

Quantifying stimulus-relevant representational drift using cross-modality contrastive learning

Representational Drift and Learning-Induced Stabilization in the Olfactory Cortex

Experience dependent plasticity of higher visual cortical areas in the mouse

Experience-dependent predictions of feedforward and contextual information in mouse visual cortex

Visual experience-dependent auditory and visual plasticity in the mouse visual cortex

Expectation violations enhance neuronal encoding of sensory information in mouse primary visual cortex

Experience-induced drift in the neural coding of individual differences in perception

Expectation violations produce error signals in mouse V1

Representational drift as the consequence of ongoing memory storage

Coherent cortical representations develop after experience via feedforward-recurrent circuit alignment

Binocular Visual Experience Drives the Maturation of Response Variability and Reliability in the Visual Cortex

Opposing Somatic and Dendritic Expression of Stimulus-Selective Response Plasticity in Mouse Primary Visual Cortex

Experience shapes activity dynamics and stimulus coding of VIP inhibitory and excitatory cells in visual cortex

Experience-Dependent Switch In Sign And Mechanisms For Plasticity In Layer 4 Of Primary Visual Cortex

Adaptation shapes the representational geometry in mouse V1 to efficiently encode the environment

Representational drift as a result of implicit regularization

Flexible neural population dynamics govern the speed and stability of sensory encoding in mouse visual cortex

Mesoscale cortical dynamics reflect the interaction of sensory evidence and temporal expectation during perceptual decision-making

Heavy-tailed statistics of cortical representational drift are advantageous for stabilised downstream readouts