Abstract:Dopamine (DA) and norepinephrine (NE) have been repeatedly implicated in neuropsychiatric vulnerability, in part via their roles in mediating the decision making processes. Although two neuromodulators share a synthesis pathway and are co-activated under states of arousal, they engage in distinct circuits and modulatory roles. However, the specific role of each neuromodulator in decision making, in particular the exploration-exploitation tradeoff, remains unclear. Revealing how each neuromodulator contributes to exploration-exploitation tradeoff is important in guiding mechanistic hypotheses emerging from computational psychiatric approaches. To understand the differences and overlaps of the roles of these two catecholamine systems in regulating exploration, a direct comparison using the same dynamic decision making task is needed. Here, we ran male and female mice in a restless two-armed bandit task, which encourages both exploration and exploitation. We systemically administered a nonselective DA antagonist (flupenthixol), a nonselective DA agonist (apomorphine), a NE beta-receptor antagonist (propranolol), and a NE beta-receptor agonist (isoproterenol), and examined changes in exploration within subjects across sessions. We found a bidirectional modulatory effect of dopamine on exploration. Increasing dopamine activity decreased exploration and decreasing dopamine activity increased exploration. The modulatory effect of beta-noradrenergic receptor activity on exploration was mediated by sex. Reinforcement learning model parameters suggested that dopamine modulation affected exploration via decision noise and norepinephrine modulation affected exploration via sensitivity to outcome. Together, these findings suggested that the mechanisms that govern the exploration-exploitation transition are sensitive to changes in both catecholamine functions and revealed differential roles for NE and DA in mediating exploration.Significance Statement Both dopamine (DA) and norepinephrine (NE) have been implicated in the decision making process. Although these two catecholamines have shared aspects of their biosynthetic pathways and projection targets, they are thought to exert many core functions via distinct neural targets and receptor subtypes. However, the computational neuroscience literature often ascribes similar roles to these catecholamines, despite the above evidence. Resolving this discrepancy is important in guiding mechanistic hypotheses emerging from computational psychiatric approaches. This study examines the role of dopamine and norepinephrine on the explore-exploit tradeoff. By testing mice, we were able to compare multiple pharmacological agents within subjects, and examine sources of individual differences, allowing direct comparison between the effects of these two catecholamines in modulating decision making.

The neurobehavioral correlates of exploration without learning: Trading off value for explicit, prospective, and variable information gains

Behavioral Bias for Exploration Is Associated with Enhanced Signaling in the Lateral and Medial Frontopolar Cortex

The Neural Correlates of Novelty and Variability in Human Decision-Making under an Active Inference Framework

Neural correlates of effort-based valuation with prospective choices

Value-free random exploration is linked to impulsivity

Learning and choosing in an uncertain world: An investigation of the explore-exploit dilemma in static and dynamic environments

Meta-Analysis Reveals That Explore-Exploit Decisions are Dissociable by Activation in the Dorsal Lateral Prefrontal Cortex and the Anterior Cingulate Cortex

Neural Signatures of Value Comparison in Human Cingulate Cortex during Decisions Requiring an Effort-Reward Trade-off

Reward-based option competition in human dorsal stream and transition from stochastic exploration to exploitation in continuous space

Decreasing Ventromedial Prefrontal Cortex Activity During Sequential Risk-Taking: An fMRI Investigation of the Balloon Analog Risk Task

Semi-orthogonal subspaces for value mediate a tradeoff between binding and generalization

Using Neural Data to Test A Theory of Investor Behavior: An Application to Realization Utility

Value, pleasure and choice in the ventral prefrontal cortex.

Overlapping and Distinct Neural Systems Code for Subjective Value during Intertemporal and Risky Decision Making

Dopamine and Norepinephrine Differentially Mediate the Exploration-exploitation Tradeoff

Gaze-centered gating, reactivation, and reevaluation of economic value in orbitofrontal cortex

PET-measured human dopamine synthesis capacity and receptor availability predict trading rewards and time-costs during foraging

Spontaneous revisitation during visual exploration as a link among strategic behavior, learning, and the hippocampus

Brain network dynamics during spontaneous strategy shifts and incremental task optimization

Brain mechanism of foraging: Reward-dependent synaptic plasticity versus neural integration of values

A novel fMRI paradigm to dissociate the behavioral and neural components of mixed-strategy decision making from non-strategic decisions in humans