Abstract:Humans have the ability to craft abstract, temporally extended and hierarchically organized plans. For instance, when considering how to make spaghetti for dinner, we typically concern ourselves with useful "subgoals" in the task, such as cutting onions, boiling pasta, and cooking a sauce, rather than particulars such as how many cuts to make to the onion, or exactly which muscles to contract. A core question is how such decomposition of a more abstract task into logical subtasks happens in the first place. Previous research has shown that humans are sensitive to a form of higher-order statistical learning named "community structure". Community structure is a common feature of abstract tasks characterized by a logical ordering of subtasks. This structure can be captured by a model where humans learn predictions of upcoming events multiple steps into the future, discounting predictions of events further away in time. One such model is the "successor representation", which has been argued to be useful for hierarchical abstraction. As of yet, no study has convincingly shown that this hierarchical abstraction can be put to use for goal-directed behavior. Here, we investigate whether participants utilize learned community structure to craft hierarchically informed action plans for goal-directed behavior. Participants were asked to search for paintings in a virtual museum, where the paintings were grouped together in "wings" representing community structure in the museum. We find that participants' choices accord with the hierarchical structure of the museum and that their response times are best predicted by a successor representation. The degree to which the response times reflect the community structure of the museum correlates with several measures of performance, including the ability to craft temporally abstract action plans. These results suggest that successor representation learning subserves hierarchical abstractions relevant for goal-directed behavior. Humans have the ability to achieve a diverse range of goals in a highly complex world. Classic theories of decision making focus on simple tasks involving single goals. In the current study, we test a recent theoretical proposal that aims to address the flexibility of hufigman decision making. By learning to predict the upcoming events, humans can acquire a 'model' of the world which they can then leverage to plan their behavior. However, given the complexity of the world, planning directly over all possible events can be overwhelming. We show that, by leveraging this predictive model, humans group similar events together into simpler "hierarchical" representations, which makes planning over these hierarchical representations markedly more efficient. Interestingly, humans seem to learn and remember both the complex predictive model and the simplified hierarchical model, using them for distinct purposes.

Updating predictions in a complex repertoire of actions and its neural representation

Neurophysiological principles underlying predictive coding during dynamic perception-action integration

The role of prediction and outcomes in adaptive cognitive control

Human and Machine Action Prediction Independent of Object Information

Modeling human activity comprehension at human scale: prediction, segmentation, and categorization

Social Constraints from an Observer's Perspective: Coordinated Actions Make an Agent's Position More Predictable.

Action Selection Based on Prediction for Robot Planning

Predicting Future Actions of Reinforcement Learning Agents

Dopamine reveals adaptive learning of actions representation

Neural Foundations of Mental Simulation: Future Prediction of Latent Representations on Dynamic Scenes

Modeling Long-horizon Tasks as Sequential Interaction Landscapes

Learning the structure of the world: The adaptive nature of state-space and action representations in multi-stage decision-making

Action Genome: Actions as Composition of Spatio-temporal Scene Graphs

Finding Islands of Predictability in Action Forecasting

What Would You Do? Acting by Learning to Predict

The interplay between executive functions and updating predictive representations

Unsupervised Learning of Effective Actions in Robotics

Predictive auxiliary objectives in deep RL mimic learning in the brain

The successor representation subserves hierarchical abstraction for goal-directed behavior

Action sequencing using visual permutations

Action-modulated midbrain dopamine activity arises from distributed control policies