Stability through plasticity: Finding robust memories through representational drift

Maanasa Natrajan,James E Fitzgerald

DOI: https://doi.org/10.1101/2024.12.19.629245

2024-12-20

Abstract:Memories are believed to be stored in synapses and retrieved through the reactivation of neural ensembles. Learning alters synaptic weights, which can interfere with previously stored memories that share the same synapses, creating a tradeoff between plasticity and stability. Interestingly, neural representations exhibit significant dynamics, even in stable environments, without apparent learning or forgetting, a phenomenon known as representational drift. Theoretical studies have suggested that multiple neural representations can correspond to a memory, with post-learning exploration of these representation solutions driving drift. However, it remains unclear whether representations explored through drift differ from those learned or offer unique advantages. Here we show that representational drift uncovers noise-robust representations that are otherwise difficult to learn. We first define the non-linear solution space manifold of synaptic weights for a fixed input-output mapping, which allows us to disentangle drift from learning and forgetting and simulate representational drift as diffusion within this manifold. Solutions explored by drift have many inactive and saturated neurons, making them robust to weight perturbations due to noise or continual learning. Such solutions are prevalent and entropically favored by drift, but their lack of gradients makes them difficult to learn and non-conducive to further learning. To overcome this, we introduce an allocation procedure that selectively shifts representations for new information into a learning-conducive regime. By combining allocation with drift, we resolve the tradeoff between learnability and robustness.

Biology