Abstract:Reinforcement learning, crucial for behavior in dynamic environments, is driven by rewards and punishments, modulated by dopamine (DA) changes. This study explores the dopaminergic system's influence on learning, particularly in Parkinson's disease (PD), where medication leads to impaired adaptability. Highlighting the role of tonic DA in signaling the valence of actions, this research investigates how DA affects response vigor and decision-making in PD. DA not only influences reward and punishment learning but also indicates the cognitive effort level and risk propensity in actions, which are essential for understanding and managing PD symptoms. In this work, we adapt our existing neurocomputational model of basal ganglia (BG) to simulate two reversal learning tasks proposed by Cools et al. We first optimized a Hebb rule for both probabilistic and deterministic reversal learning, conducted a sensitivity analysis (SA) on parameters related to DA effect, and compared performances between three groups: PD-ON, PD-OFF, and control subjects. In our deterministic task simulation, we explored switch error rates after unexpected task switches and found a U-shaped relationship between tonic DA levels and switch error frequency. Through SA, we classify these three groups. Then, assuming that the valence of the stimulus affects the tonic levels of DA, we were able to reproduce the results by Cools et al. As for the probabilistic task simulation, our results are in line with clinical data, showing similar trends with PD-ON, characterized by higher tonic DA levels that are correlated with increased difficulty in both acquisition and reversal tasks. Our study proposes a new hypothesis: valence, signaled by tonic DA levels, influences learning in PD, confirming the uncorrelation between phasic and tonic DA changes. This hypothesis challenges existing paradigms and opens new avenues for understanding cognitive processes in PD, particularly in reversal learning tasks.

A Causal Role for the Pedunculopontine Nucleus in Human Instrumental Learning

Hemispheric dissociation of reward processing in humans: insights from deep brain stimulation

Nucleus accumbens dopamine release reflects Bayesian inference during instrumental learning

Human Substantia Nigra Neurons Encode Reward Expectations

Subsecond fluctuations in extracellular dopamine encode reward and punishment prediction errors in humans

A causal link between prediction errors, dopamine neurons and learning

A novel critic signal in identified midbrain dopaminergic neurons of mice training in operant tasks

How Instructed Knowledge Modulates the Neural Systems of Reward Learning

Model-based reward prediction in the primate prefrontal cortex

Modulation of neural activity in frontopolar cortex drives reward-based motor learning

Reward Prediction Error in Learning-Related Behaviors

Prelimbic cortex ensembles promote appetitive learning-associated behavior

Dopamine-dependent reinforcement of motor skill learning: evidence from Gilles de la Tourette syndrome

Valence-dependent dopaminergic modulation during reversal learning in Parkinson's disease: A neurocomputational approach

Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry

Encoding Motivation Prediction Errors in the Human Dopaminergic Reward System

The Neural Dynamics of Reward Value and Risk Coding in the Human Orbitofrontal Cortex.

Pre-existing visual responses in a projection-defined dopamine population explain individual learning trajectories

Role of Dopamine D2 Receptors in Human Reinforcement Learning.

Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning