Abstract:The neurotransmitter dopamine is implicated in diverse functions, including reward processing, reinforcement learning, and cognitive control. The tendency to discount future rewards over time has long been discussed in the context of potential dopaminergic modulation. Here we examined the effect of a single dose of the D2 receptor antagonist haloperidol (2 mg) on temporal discounting in healthy female and male human participants. Our approach extends previous pharmacological studies in two ways. First, we applied combined temporal discounting drift diffusion models to examine choice dynamics. Second, we examined dopaminergic modulation of reward magnitude effects on temporal discounting. Hierarchical Bayesian parameter estimation revealed that the data were best accounted for by a temporal discounting drift diffusion model with nonlinear trialwise drift rate scaling. This model showed good parameter recovery, and posterior predictive checks revealed that it accurately reproduced the relationship between decision conflict and response times in individual participants. We observed reduced temporal discounting and substantially faster nondecision times under haloperidol compared with placebo. Discounting was steeper for low versus high reward magnitudes, but this effect was largely unaffected by haloperidol. Results were corroborated by model-free analyses and modeling via more standard approaches. We previously reported elevated caudate activation under haloperidol in this sample of participants, supporting the idea that haloperidol elevated dopamine neurotransmission (e.g., by blocking inhibitory feedback via presynaptic D2 auto-receptors). The present results reveal that this is associated with an augmentation of both lower-level (nondecision time) and higher-level (temporal discounting) components of the decision process.SIGNIFICANCE STATEMENT Dopamine is implicated in reward processing, reinforcement learning, and cognitive control. Here we examined the effects of a single dose of the D2 receptor antagonist haloperidol on temporal discounting and choice dynamics during the decision process. We extend previous studies by applying computational modeling using the drift diffusion model, which revealed that haloperidol reduced the nondecision time and reduced impulsive choice compared with placebo. These findings are compatible with a haloperidol-induced increase in striatal dopamine (e.g., because of a presynaptic mechanism). Our data provide novel insights into the contributions of dopamine to value-based decision-making and highlight how comprehensive model-based analyses using sequential sampling models can inform the effects of pharmacological modulation on choice processes.

The drift diffusion model as the choice rule in reinforcement learning

The drift diffusion model as the choice rule in inter-temporal and risky choice: a case study in medial orbitofrontal cortex lesion patients and controls

A reinforcement learning diffusion decision model for value-based decisions

An Overcomplete Approach to Fitting Drift-Diffusion Decision Models to Trial-By-Trial Data

HMM for Discovering Decision-Making Dynamics Using Reinforcement Learning Experiments

The role of reinforcement learning in shaping the decision policy in methamphetamine use disorders

Learning to Choose: Behavioral Dynamics Underlying the Initial Acquisition of Decision-Making

Learning to Choose: Behavioral Dynamics Underlying the Initial Acquisition of Decision Making

The Tweedledum and Tweedledee of dynamic decisions: Discriminating between diffusion decision and accumulator models

Decision-Focused Model-based Reinforcement Learning for Reward Transfer

Using Drift Diffusion and RL Models to Disentangle Effects of Depression On Decision-Making vs. Learning in the Probabilistic Reward Task

A practical introduction to using the drift diffusion model of decision-making in cognitive psychology, neuroscience, and health sciences

Behavior Decision of Mobile Robot With a Neurophysiologically Motivated Reinforcement Learning Model

Unifying and generalizing models of neural dynamics during decision-making

Interpreting the parameters of the diffusion model: An empirical validation

Testing the drift-diffusion model

Support for the Time-Varying Drift Rate Model of Perceptual Discrimination in Dynamic and Static Noise Using Bayesian Model-Fitting Methodology

Dopaminergic Modulation of Human Intertemporal Choice: A Diffusion Model Analysis Using the D2-Receptor Antagonist Haloperidol

Brain Mechanisms for Perceptual and Reward-Related Decision-Making

Informing cognitive abstractions through neuroimaging: The neural drift diffusion model.

Neural Substrates of the Drift-Diffusion Model in Brain Disorders