Abstract:Reinforcement learning, crucial for behavior in dynamic environments, is driven by rewards and punishments, modulated by dopamine (DA) changes. This study explores the dopaminergic system's influence on learning, particularly in Parkinson's disease (PD), where medication leads to impaired adaptability. Highlighting the role of tonic DA in signaling the valence of actions, this research investigates how DA affects response vigor and decision-making in PD. DA not only influences reward and punishment learning but also indicates the cognitive effort level and risk propensity in actions, which are essential for understanding and managing PD symptoms. In this work, we adapt our existing neurocomputational model of basal ganglia (BG) to simulate two reversal learning tasks proposed by Cools et al. We first optimized a Hebb rule for both probabilistic and deterministic reversal learning, conducted a sensitivity analysis (SA) on parameters related to DA effect, and compared performances between three groups: PD-ON, PD-OFF, and control subjects. In our deterministic task simulation, we explored switch error rates after unexpected task switches and found a U-shaped relationship between tonic DA levels and switch error frequency. Through SA, we classify these three groups. Then, assuming that the valence of the stimulus affects the tonic levels of DA, we were able to reproduce the results by Cools et al. As for the probabilistic task simulation, our results are in line with clinical data, showing similar trends with PD-ON, characterized by higher tonic DA levels that are correlated with increased difficulty in both acquisition and reversal tasks. Our study proposes a new hypothesis: valence, signaled by tonic DA levels, influences learning in PD, confirming the uncorrelation between phasic and tonic DA changes. This hypothesis challenges existing paradigms and opens new avenues for understanding cognitive processes in PD, particularly in reversal learning tasks.

The Neuropsychology of Ventral Prefrontal Cortex: Decision-making and Reversal Learning

Decision making and reward in frontal cortex: Complementary evidence from neurophysiological and neuropsychological studies.

Decision making and neuropsychiatry

Decision-making processes following damage to the prefrontal cortex.

Defining the Neural Mechanisms of Probabilistic Reversal Learning Using Event-Related Functional Magnetic Resonance Imaging

Monoaminergic-Dependent Cognitive Functions of the Prefrontal Cortex in Monkey and Man

Prefrontal Executive and Cognitive Functions in Rodents: Neural and Neurochemical Substrates

Impairments in an early stage of the decision-making process in patients with ventromedial prefrontal damage: preliminary results

Frontal Cortex and Reward-Guided Learning and Decision-Making

Valence-dependent dopaminergic modulation during reversal learning in Parkinson's disease: A neurocomputational approach

Effects of Orbitofrontal, Infralimbic and Prelimbic Cortical Lesions on Serial Spatial Reversal Learning in the Rat.

Neural correlates of rapid reversal learning in a simple model of human social interaction.

Differential Contributions of the Primate Ventrolateral Prefrontal and Orbitofrontal Cortex to Serial Reversal Learning

Memory, Decision-Making, and the Ventromedial Prefrontal Cortex (vmPFC): The Roles of Subcallosal and Posterior Orbitofrontal Cortices in Monitoring and Control Processes

Transforming brain signals related to value evaluation and self-control into behavioral choices.

The Neurochemical Modulation of Prefrontal Control Processes

Neurobiological foundations and clinical relevance of effort-based decision-making

The Contributions of Lesion Laterality and Lesion Volume to Decision-Making Impairment Following Frontal Lobe Damage.

Specializations for reward-guided decision-making in the primate ventral prefrontal cortex

Optimal decision making and the anterior cingulate cortex

The neural substrates of probabilistic and intertemporal decision making