Abstract:Humans have the ability to craft abstract, temporally extended and hierarchically organized plans. For instance, when considering how to make spaghetti for dinner, we typically concern ourselves with useful "subgoals" in the task, such as cutting onions, boiling pasta, and cooking a sauce, rather than particulars such as how many cuts to make to the onion, or exactly which muscles to contract. A core question is how such decomposition of a more abstract task into logical subtasks happens in the first place. Previous research has shown that humans are sensitive to a form of higher-order statistical learning named "community structure". Community structure is a common feature of abstract tasks characterized by a logical ordering of subtasks. This structure can be captured by a model where humans learn predictions of upcoming events multiple steps into the future, discounting predictions of events further away in time. One such model is the "successor representation", which has been argued to be useful for hierarchical abstraction. As of yet, no study has convincingly shown that this hierarchical abstraction can be put to use for goal-directed behavior. Here, we investigate whether participants utilize learned community structure to craft hierarchically informed action plans for goal-directed behavior. Participants were asked to search for paintings in a virtual museum, where the paintings were grouped together in "wings" representing community structure in the museum. We find that participants' choices accord with the hierarchical structure of the museum and that their response times are best predicted by a successor representation. The degree to which the response times reflect the community structure of the museum correlates with several measures of performance, including the ability to craft temporally abstract action plans. These results suggest that successor representation learning subserves hierarchical abstractions relevant for goal-directed behavior. Humans have the ability to achieve a diverse range of goals in a highly complex world. Classic theories of decision making focus on simple tasks involving single goals. In the current study, we test a recent theoretical proposal that aims to address the flexibility of hufigman decision making. By learning to predict the upcoming events, humans can acquire a 'model' of the world which they can then leverage to plan their behavior. However, given the complexity of the world, planning directly over all possible events can be overwhelming. We show that, by leveraging this predictive model, humans group similar events together into simpler "hierarchical" representations, which makes planning over these hierarchical representations markedly more efficient. Interestingly, humans seem to learn and remember both the complex predictive model and the simplified hierarchical model, using them for distinct purposes.

Humans rationally balance detailed and temporally abstract world models

Trial-by-trial learning of successor representations in human behavior

The successor representation subserves hierarchical abstraction for goal-directed behavior

Neural Mechanisms of Human Decision-Making

Modeling Human Behavior Part II -- Cognitive approaches and Uncertainty

Humans adaptively resolve the explore-exploit dilemma under cognitive constraints: Evidence from a multi-armed bandit task

Abstract representations of events arise from mental errors in learning and memory

Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making

Using recurrent neural network to estimate irreducible stochasticity in human choice behavior

Modeling sensory-motor decisions in natural behavior

Simple Plans or Sophisticated Habits? State, Transition and Learning Interactions in the Two-Step Task

Harnessing the flexibility of neural networks to predict dynamic theoretical parameters underlying human choice behavior

Learning World Models With Hierarchical Temporal Abstractions: A Probabilistic Perspective

Short-term Memory Traces for Action Bias in Human Reinforcement Learning

The Neural Correlates of Novelty and Variability in Human Decision-Making under an Active Inference Framework

Competing neural representations of choice shape evidence accumulation in humans

Policy adjustment in a dynamic economic game

Ongoing, rational calibration of reward-driven perceptual biases

Cognitive mechanisms of learning in sequential decision-making under uncertainty: an experimental and theoretical approach

Discovering Cognitive Strategies with Tiny Recurrent Neural Networks

Capturing Optimal and Suboptimal behavior Of Agents Via Structure Learning Of Their Internal Model