Abstract:Humans have the ability to craft abstract, temporally extended and hierarchically organized plans. For instance, when considering how to make spaghetti for dinner, we typically concern ourselves with useful "subgoals" in the task, such as cutting onions, boiling pasta, and cooking a sauce, rather than particulars such as how many cuts to make to the onion, or exactly which muscles to contract. A core question is how such decomposition of a more abstract task into logical subtasks happens in the first place. Previous research has shown that humans are sensitive to a form of higher-order statistical learning named "community structure". Community structure is a common feature of abstract tasks characterized by a logical ordering of subtasks. This structure can be captured by a model where humans learn predictions of upcoming events multiple steps into the future, discounting predictions of events further away in time. One such model is the "successor representation", which has been argued to be useful for hierarchical abstraction. As of yet, no study has convincingly shown that this hierarchical abstraction can be put to use for goal-directed behavior. Here, we investigate whether participants utilize learned community structure to craft hierarchically informed action plans for goal-directed behavior. Participants were asked to search for paintings in a virtual museum, where the paintings were grouped together in "wings" representing community structure in the museum. We find that participants' choices accord with the hierarchical structure of the museum and that their response times are best predicted by a successor representation. The degree to which the response times reflect the community structure of the museum correlates with several measures of performance, including the ability to craft temporally abstract action plans. These results suggest that successor representation learning subserves hierarchical abstractions relevant for goal-directed behavior. Humans have the ability to achieve a diverse range of goals in a highly complex world. Classic theories of decision making focus on simple tasks involving single goals. In the current study, we test a recent theoretical proposal that aims to address the flexibility of hufigman decision making. By learning to predict the upcoming events, humans can acquire a 'model' of the world which they can then leverage to plan their behavior. However, given the complexity of the world, planning directly over all possible events can be overwhelming. We show that, by leveraging this predictive model, humans group similar events together into simpler "hierarchical" representations, which makes planning over these hierarchical representations markedly more efficient. Interestingly, humans seem to learn and remember both the complex predictive model and the simplified hierarchical model, using them for distinct purposes.

The brain hierarchically represents the past and future during multistep anticipation

Consolidation Enhances Sequential Multistep Anticipation but Diminishes Access to Perceptual Features

The capacity of the medial temporal lobe to represent memory items in their ordinal position in a sequence is domain-general

Memory, Space, and Planning: Multiscale Predictive Representations

Anticipated moments: temporal structure in attention

Time Regained: How the Human Brain Constructs Memory for Time

Human hippocampal and entorhinal neurons encode the temporal structure of experience

Spatiotemporal brain hierarchies of auditory memory recognition and predictive coding

Time for Memories

Temporal asymmetry of neural representations predicts memory decisions

A preserved neural code for temporal order between memory formation and recall in the human medial temporal lobe

The medial prefrontal cortex leaves the hippocampus when it prepares for the future

Sequence Structure Organizes Items in Varied Latent States of Working Memory Neural Network.

2-D Neural Geometry Underpins Hierarchical Organization of Sequence in Human Working Memory

Temporal dynamics of competition between statistical learning and episodic memory in intracranial recordings of human visual cortex

Evidence for the incorporation of temporal duration information in human hippocampal long-term memory sequence representations

The successor representation subserves hierarchical abstraction for goal-directed behavior

Fast-backward Replay of Sequentially Memorized Items in Humans.

A model of how hierarchical representations constructed in the hippocampus are used to navigate through space

What comes first? Electrophysiological differences in the temporal course of memory and future thinking

Two-dimensional neural geometry underpins hierarchical organization of sequence in human working memory