Abstract:<p>This article aims to provide a synthesis on the question how brain structures cooperate to accomplish hierarchically organized behaviors, characterized by low‐level, habitual routines nested in larger sequences of planned, goal‐directed behavior. The functioning of a connected set of brain structures—prefrontal cortex, hippocampus, striatum, and dopaminergic mesencephalon—is reviewed in relation to two important distinctions: (a) goal‐directed as opposed to habitual behavior and (b) model‐based and model‐free learning. Recent evidence indicates that the orbitomedial prefrontal cortices not only subserve goal‐directed behavior and model‐based learning, but also code the "landscape" (task space) of behaviorally relevant variables. While the hippocampus stands out for its role in coding and memorizing world state representations, it is argued to function in model‐based learning but is not required for coding of action–outcome contingencies, illustrating that goal‐directed behavior is not congruent with model‐based learning. While the dorsolateral and dorsomedial striatum largely conform to the dichotomy between habitual versus goal‐directed behavior, ventral striatal functions go beyond this distinction. Next, we contextualize findings on coding of reward‐prediction errors by ventral tegmental dopamine neurons to suggest a broader role of mesencephalic dopamine cells, viz. in behavioral reactivity and signaling unexpected sensory changes. We hypothesize that goal‐directed behavior is hierarchically organized in interconnected cortico‐basal ganglia loops, where a limbic‐affective prefrontal‐ventral striatal loop controls action selection in a dorsomedial prefrontal–striatal loop, which in turn regulates activity in sensorimotor‐dorsolateral striatal circuits. This structure for behavioral organization requires alignment with mechanisms for memory formation and consolidation. We propose that frontal corticothalamic circuits form a high‐level loop for memory processing that initiates and temporally organizes nested activities in lower‐level loops, including the hippocampus and the ripple‐associated replay it generates. The evidence on hierarchically organized behavior converges with that on consolidation mechanisms in suggesting a frontal‐to‐caudal directionality in processing control.</p>

Optogenetic Editing Reveals the Hierarchical Organization of Learned Action Sequences

Complementary Contributions of Striatal Projection Pathways to Action Initiation and Execution

Contributions of the basal ganglia to action sequence learning and performance

Multiple dynamic interactions from basal ganglia direct and indirect pathways mediate action selection

Basal Ganglia Subcircuits Distinctively Encode the Parsing and Concatenation of Action Sequences.

Nigrostriatal Dopamine Signals Sequence-Specific Action-Outcome Prediction Errors

Start/stop Signals Emerge in Nigrostriatal Circuits During Sequence Learning.

Learning, memory and consolidation mechanisms for behavioral control in hierarchically organized cortico‐basal ganglia systems

Adaptation of sequential action benefits from timing variability related to lateral basal ganglia circuitry

Rethinking dopamine‐guided action sequence learning

Asymmetric Cortical Projections to Striatal Direct and Indirect Pathways Distinctly Control Actions

The Contribution of Premotor Cortico-Striatal Projections to the Execution of Serial Order Sequences

Protosequences in human cortical organoids model intrinsic states in the developing cortex

Refinement of efficient encodings of movement in the dorsolateral striatum throughout learning

Direct Pathway Neurons in the Mouse Ventral Striatum Are Active During Goal-Directed Action but Not Reward Consumption During Operant Conditioning

Striatal modulation supports policy-specific reinforcement and not action selection

Motor sequence learning involves better prediction of the next action and optimization of movement trajectories

Striatopallidal Pathway Distinctly Modulates Goal-Directed Valuation and Acquisition of Instrumental Behavior Via Striatopallidal Output Projections

Neural dynamics hierarchy in motor cortex and striatum across naturalistic behaviors

An iterative neural processing sequence orchestrates feeding

Neural Sequences Underlying Directed Turning in C. elegans