Abstract:Cognitive flexibility refers to the ability to adjust to changes in the environment and is essential for adaptive behavior. It can be investigated using laboratory tests such as probabilistic reversal learning (PRL). In individuals with both Cocaine Use Disorder (CUD) and Gambling Disorder (GD), overall impairments in PRL flexibility are observed. However, it is poorly understood whether this impairment depends on the same brain mechanisms in cocaine and gambling addictions. Reinforcement learning (RL) is the process by which rewarding or punishing feedback from the environment is used to adjust behavior, to maximise reward and minimise punishment. Using RL models, a deeper mechanistic explanation of the latent processes underlying cognitive flexibility can be gained. Here, we report results from a re-analysis of PRL data from control participants (n=18) and individuals with either GD (n=18) or CUD (n=20) using a hierarchical Bayesian RL approach. We observed significantly reduced ‘stimulus stickiness’ (i.e., stimulus-bound perseveration) in GD, which may reflect increased exploratory behavior that is insensitive to outcomes. RL parameters were unaffected in CUD. We relate the behavioral findings to their underlying neural substrates through an analysis of task-based fMRI data. We report differences in tracking reward and punishment expected values (EV) in individuals with GD compared to controls, with greater activity during reward EV tracking in the cingulate gyrus and amygdala. In CUD, we observed reduced responses to positive punishment prediction errors (PPE) and increased activity following negative PPEs in the superior frontal gyrus compared to controls. Thus, an RL framework serves to differentiate behavior in a probabilistic learning paradigm in two compulsive disorders, GD and CUD. ### Competing Interest Statement The Centre for Gambling Research at UBC (L.C.) is funded by support from the Province of British Columbia and the British Columbia Lottery Corporation. J.W.D. has received research grants from Boehringer Ingelheim Pharma GmbH and GlaxoSmithKline and receives royalties from Springer Verlag. T.W.R. discloses consultancy with Cambridge Cognition; he receives editorial honoraria from Springer-Nature and Elsevier and a research grant from Shionogi. R.N.C. consults for Campden Instruments and receives royalties from Cambridge Enterprise, Routledge, and Cambridge University Press.

Quantum Reinforcement Learning During Human Decision-Making.

A Subjective Model of Human Decision Making Based on Quantum Decision Theory

Brain-Inspired Agents for Quantum Reinforcement Learning

A model of quantum-like decision-making with applications to psychology and cognitive science

Efficient quantum recurrent reinforcement learning via quantum reservoir computing

An Introduction to Quantum Reinforcement Learning (QRL)

Quantum Computation For Action Selection Using Reinforcement Learning

Unified Models of Human Behavioral Agents in Bandits, Contextual Bandits and RL

A quantum-classical reinforcement learning model to play Atari games

Quantum-Cognitive Neural Networks: Assessing Confidence and Uncertainty with Human Decision-Making Simulations

Modeling Human Decision-making: An Overview of the Brussels Quantum Approach

Cognitive Models as Simulators: The Case of Moral Decision-Making

Dissociating Value-Based Neurocomputation from Subsequent Selection-Related Activations in Human Decision-Making

The neural substrates of probabilistic and intertemporal decision making

On the convergence of projective-simulation-based reinforcement learning in Markov decision processes

Computational modeling of reinforcement learning and functional neuroimaging of probabilistic reversal dissociates compulsive behaviors in Gambling and Cocaine Use Disorders

QuLBIT: Quantum-Like Bayesian Inference Technologies for Cognition and Decision

Reconciling Irrational Human Behavior with AI based Decision Making: A Quantum Probabilistic Approach

HMM for Discovering Decision-Making Dynamics Using Reinforcement Learning Experiments

A reinforcement learning diffusion decision model for value-based decisions

Quantum Machine Learning Architecture Search via Deep Reinforcement Learning