Abstract:Reward-associated cues serve different functions depending on whether they precede or terminate action sequences. Cues that precede action sequences and signal opportunity for reward could serve as GO signals to initiate the sequence, whereas sequence termination cues could serve as response feedback by signaling reward delivery. Reward expectation during sequence execution depends on these cues and might condition whether behavior is habitual or goal-directed. However, it remains unknown how sequence initiation and termination cues differentially affect reward expectation and contribute to habit learning. Further, while mesolimbic dopamine plays a key role in cue-induced reward expectation and sequence learning, how dynamic changes in dopamine signals differ depending on the response strategy is unclear. Here, we determined how mesolimbic DA signals change over training during cue-mediated sequence learning, depending on the type of cue and the nature of behavioral control. We found sequence initiation and termination cues differentially affect reward expectation during action sequences, with the termination cue contributing to habit and automaticity. Distinct response strategies induced by sequence initiation and termination cues induced differential changes in mesolimbic DA signals that captured variations in reward expectation along sequence execution. Notably, habit-like behavior induced by the sequence termination cue was associated with a rapid shift in DA signals from reward retrieval to the cue. This habit-like behavior was reflected in behavioral inflexibility and attenuated DA reward prediction error signals. Finally, using optogenetics, we provide evidence that phasic DA activity elicited by the sequence termination cue is critical for the development of habit-like behavior.

Slowly Evolving Dopaminergic Activity Modulates the Moment-to-moment Probability of Reward-Related Self-Timed Movements

Dopamine dynamics are dispensable for movement but promote reward responses

Intrinsic dopamine and acetylcholine dynamics in the striatum of mice

Nigrostriatal Dopamine Signals Sequence-Specific Action-Outcome Prediction Errors

Reinforcement Learning Links Spontaneous Cortical Dopamine Impulses to Reward

Role of dopamine in reward expectation and predictability during execution of action sequences

Intrinsic Reward-Like Dopamine and Acetylcholine Dynamics in Striatum

Dopamine transients follow a striatal gradient of reward time horizons

Dopamine dynamics in nucleus accumbens across reward-based learning of goal-directed whisker-to-lick sensorimotor transformations in mice

Dopamine–endocannabinoid Interactions Mediate Spike-Timing-dependent Potentiation in the Striatum

Dynamic Nigrostriatal Dopamine Biases Action Selection.

Acetylcholine demixes heterogeneous dopamine signals for learning and moving

Behavioral encoding across timescales by region-specific dopamine dynamics

A mismatch between striatal cholinergic pauses and dopaminergic reward prediction errors

A novel critic signal in identified midbrain dopaminergic neurons of mice training in operant tasks

Dopamine Increases Accuracy and Lengthens Deliberation Time in Explicit Motor Skill Learning

Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward

Neural mechanisms underlying uninstructed orofacial movements during reward-based learning behaviors

Striatal dopamine signals reflect perceived cue–action–outcome associations in mice

Does phasic dopamine release cause policy updates?

Dopamine alters motor learning performance in the presence and absence of feedback