Abstract:Everyday, we encounter situations where available choices are nearly equally rewarding (high conflict) calling for some tough decision making. Experimental recordings showed that the activity of Sub Thalamic Nucleus (STN) increases during such situations providing the extra time needed to make the right decision, teasing apart the most rewarding choice from the runner up closely trailing behind. This prolonged deliberation necessary for decision making under high conflict was absent in Parkinson's disease (PD) patients who underwent Deep Brain Stimulation (DBS) surgery of STN. In an attempt to understand the underlying cause of such adverse response, we built a 2D spiking network model (50 × 50 lattice) of Basal ganglia incorporating the key nuclei. Using the model we studied the Probabilistic learning task (PLT) in untreated, treated (L-Dopa and Dopamine Agonist) and STN-DBS PD conditions. Based on the experimental observation that dopaminergic activity is analogous to temporal difference (TD) and induces cortico-striatal plasticity, we introduced learning in the cortico-striatal weights. The results show that healthy and untreated conditions of PD model were able to more or less equally select (avoid) the rewarding (punitive) choice, a behavior that was absent in treated PD condition. The time taken to select a choice in high conflict trials was high in normal condition, which is in agreement with experimental results. The treated PD (Dopamine Agonist) patients made impulsive decisions (small reaction time) which in turn led to poor performance. The underlying cause of the observed impulsivity in DBS patients was studied in the model by (1) varying the electrode position within STN, (2) causing antidromic activation of GPe neurons. The effect of electrode position on reaction time was analyzed by studying the activity of STN neurons where, a decrease in STN neural activity was observed for certain electrode positions. We also observed that a higher antidromic activation of GPe neurons does not impact the learning ability but decreases reaction time as reported in DBS patients. These results suggest a probable role of electrode and antidromic activation in modulating the STN activity and eventually affecting the patient's performance on PLT.

Using TD learning to simulate working memory performance in a model of the prefrontal cortex and basal ganglia

Adaptive coordination of working-memory and reinforcement learning in non-human primates performing a trial-and-error problem solving task

How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis

A Basal Ganglia model for understanding working memory functions in healthy and Parkinson's conditions

Neuronal Representation of a Working Memory-Based Decision Strategy in the Motor and Prefrontal Cortico-Basal Ganglia Loops

Rehearsing Biological Motion in Working Memory: an EEG Study.

A Neurocomputational Theory of the Dopaminergic Modulation of Working Memory Functions

Modeling working memory: a computational implementation of the Time-Based Resource-Sharing theory

Probing the Role of Medication, DBS Electrode Position, and Antidromic Activation on Impulsivity Using a Computational Model of Basal Ganglia

Timing-dependent modulation of working memory by VTA dopamine release in medial prefrontal cortex

Reward learning and working memory: Effects of massed versus spaced training and post-learning delay period

The Effect of Dopamine on Working Memory.

Prefrontal Neurons Encode Actions and Outcomes in Conjunction with Spatial Location in Rats Performing a Dynamic Delayed Non-Match to Position Task

Efficient Reinforcement Learning of a Reservoir Network Model of Parametric Working Memory Achieved with a Cluster Population Winner-Take-all Readout Mechanism.

Dorsolateral prefrontal cortex mediates working memory processes in motor skill learning

Working memory and response selection: a computational account of interactions among cortico-basalganglio-thalamic loops

Representation and Timing in Theories of the Dopamine System

Attention and Working Memory: a Dynamical Model of Neuronal Activity in the Prefrontal Cortex

Using Neural Networks to Uncover the Relationship between Highly Variable Behavior and EEG during a Working Memory Task with Distractors

Long-term learning transforms prefrontal cortex selectivity during working memory