Abstract:The mammalian hippocampus contains a cognitive map that represents an animal's position in the environment and generates offline "replay" for the purposes of recall, planning, and forming long term memories. Recently, it's been found that artificial neural networks trained to predict sensory inputs develop spatially tuned cells, aligning with predictive theories of hippocampal function. However, whether predictive learning can also account for the ability to produce offline replay is unknown. Here, we find that spatially tuned cells, which robustly emerge from all forms of predictive learning, do not guarantee the presence of a cognitive map with the ability to generate replay. Offline simulations only emerged in networks that used recurrent connections and head-direction information to predict multi-step observation sequences, which promoted the formation of a continuous attractor reflecting the geometry of the environment. These offline trajectories were able to show wake-like statistics, autonomously replay recently experienced locations, and could be directed by a virtual head direction signal. Further, we found that networks trained to make cyclical predictions of future observation sequences were able to rapidly learn a cognitive map and produced sweeping representations of future positions reminiscent of hippocampal theta sweeps. These results demonstrate how hippocampal-like representation and replay can emerge in neural networks engaged in predictive learning, and suggest that hippocampal theta sequences reflect a circuit that implements a data-efficient algorithm for sequential predictive learning. Together, this framework provides a unifying theory for hippocampal functions and hippocampal-inspired approaches to artificial intelligence.

A model of hippocampal replay driven by experience and environmental structure facilitates spatial learning

A Robotic Model of Hippocampal Reverse Replay for Reinforcement Learning

Geometry of Spatial Memory Replay

Model-Based and Model-Free Replay Mechanisms for Reinforcement Learning in Neurorobotics

A Brain-Inspired Model of Hippocampal Spatial Cognition Based on a Memory-Replay Mechanism

Replays of spatial memories suppress topological fluctuations in cognitive map

Experience Replay Facilitates the Rapid Formation of Multi-Scale Hexagonal Patterns of Grid Cells

The role of experience in prioritizing hippocampal replay

Prioritizing replay when future goals are unknown

Prioritized Sweeping Neural DynaQ with Multiple Predecessors, and Hippocampal Replays

Coherent noise enables probabilistic sequence replay in spiking neuronal networks

Learning predictive cognitive maps with spiking neurons during behavior and replays

Experience replay facilitates the formation of the hexagonal pattern of grid cells

Paradoxical replay can protect contextual task representations from destructive interference when experience is unbalanced

Sequential predictive learning is a unifying theory for hippocampal representation and replay

Constructing future behaviour in the hippocampal formation through composition and replay

Distinct replay signatures for prospective decision-making and memory preservation

Brain-Like Replay Naturally Emerges in Reinforcement Learning Agents

Hippocampal replay sequence governed by spontaneous brain-wide dynamics

Hippocampal sharp wave-ripples and the associated sequence replay emerge from structured synaptic interactions in a network model of area CA3

A unifying account of replay as context-driven memory reactivation