Abstract:The predictive map hypothesis is a promising candidate principle for hippocampal function. A favoured formalisation of this hypothesis, called the successor representation, proposes that each place cell encodes the expected state occupancy of its target location in the near future. This predictive framework is supported by behavioural as well as electrophysiological evidence and has desirable consequences for both the generalisability and efficiency of reinforcement learning algorithms. However, it is unclear how the successor representation might be learnt in the brain. Error-driven temporal difference learning, commonly used to learn successor representations in artificial agents, is not known to be implemented in hippocampal networks. Instead, we demonstrate that spike-timing dependent plasticity (STDP), a form of Hebbian learning, acting on temporally compressed trajectories known as 'theta sweeps', is sufficient to rapidly learn a close approximation to the successor representation. The model is biologically plausible - it uses spiking neurons modulated by theta-band oscillations, diffuse and overlapping place cell-like state representations, and experimentally matched parameters. We show how this model maps onto known aspects of hippocampal circuitry and explains substantial variance in the temporal difference successor matrix, consequently giving rise to place cells that demonstrate experimentally observed successor representation-related phenomena including backwards expansion on a 1D track and elongation near walls in 2D. Finally, our model provides insight into the observed topographical ordering of place field sizes along the dorsal-ventral axis by showing this is necessary to prevent the detrimental mixing of larger place fields, which encode longer timescale successor representations, with more fine-grained predictions of spatial location.

Sequential predictive learning is a unifying theory for hippocampal representation and replay

Predictive Sequence Learning in the Hippocampal Formation

Learning predictive cognitive maps with spiking neurons during behavior and replays

A model of hippocampal replay driven by experience and environmental structure facilitates spatial learning

Hippocampal representations emerge when training recurrent neural networks on a memory dependent maze navigation task

Coherent noise enables probabilistic sequence replay in spiking neuronal networks

Sequence learning, prediction, and replay in networks of spiking neurons

The hippocampus as a predictive map

Rapid learning of predictive maps with STDP and theta phase precession

Neural learning rules for generating flexible predictions and computing the successor representation

Spatial Memory Sequence Encoding and Replay During Modeled Theta and Ripple Oscillations

Hippocampal replay sequence governed by spontaneous brain-wide dynamics

A hippocampus CA3 spiking neural network model for storage and retrieval of sequential memory

Replay in human visual cortex is linked to the formation of successor representations and independent of consciousness

Associative and predictive hippocampal codes support memory-guided behaviors

Predictive learning rules generate a cortical-like replay of probabilistic sensory experiences

A Robotic Model of Hippocampal Reverse Replay for Reinforcement Learning

Prioritized Sweeping Neural DynaQ with Multiple Predecessors, and Hippocampal Replays

Replays of spatial memories suppress topological fluctuations in cognitive map

Development of Hippocampal Replay with Learning in Spatial Memory of Precise Goal Locations

Reversed and forward buffering of behavioral spike sequences enables retrospective and prospective retrieval in hippocampal regions CA3 and CA1