Abstract:Learning to make adaptive decisions involves making choices, assessing their consequence, and leveraging this assessment to attain higher rewarding states. Despite vast literature on value-based decision-making, relatively little is known about the cognitive processes underlying decisions in highly uncertain contexts. Real world decisions are rarely accompanied by immediate feedback, explicit rewards, or complete knowledge of the environment. Being able to make informed decisions in such contexts requires significant knowledge about the environment, which can only be gained via exploration. Here we aim at understanding and formalizing the brain mechanisms underlying these processes. To this end, we first designed and performed an experimental task. Human participants had to learn to maximize reward while making sequences of decisions with only basic knowledge of the environment, and in the absence of explicit performance cues. Participants had to rely on their own internal assessment of performance to reveal a covert relationship between their choices and their subsequent consequences to find a strategy leading to the highest cumulative reward. Our results show that the participants' reaction times were longer whenever the decision involved a future consequence, suggesting greater introspection whenever a delayed value had to be considered. The learning time varied significantly across participants. Second, we formalized the neurocognitive processes underlying decision-making within this task, combining mean-field representations of competing neural populations with a reinforcement learning mechanism. This model provided a plausible characterization of the brain dynamics underlying these processes, and reproduced each aspect of the participants' behavior, from their reaction times and choices to their learning rates. In summary, both the experimental results and the model provide a principled explanation to how delayed value may be computed and incorporated into the neural dynamics of decision-making, and to how learning occurs in these uncertain scenarios.

Dual-process modeling of sequential decision making in the balloon analogue risk task

Distinctive Neural Substrates of low and high Risky Decision Making: Evidence from the Balloon Analog Risk Task

Sequential Decision Learning Models with Balloon Analogy Risk Task

Embodied sequential sampling models and dynamic neural fields for decision-making: Why hesitate between two when a continuum is the answer

Modeling Decision-Making in Schizophrenia: Associations Between Computationally Derived Risk Propensity and Self-Reported Risk Perception

Common and distinct neural correlates of intertemporal and risky decision-making: Meta-analytical evidence for the dual-system theory

Now for sure or later with a risk? Modeling risky intertemporal choice as accumulated preference.

Dissociating Value-Based Neurocomputation from Subsequent Selection-Related Activations in Human Decision-Making

Cognitive mechanisms of learning in sequential decision-making under uncertainty: an experimental and theoretical approach

The neural substrates of probabilistic and intertemporal decision making

Overt visual attention and value computation in complex risky choice

Multiple Systems in Decision Making

Navigation through the complex world -- the neurophysiology of decision-making processes

Risk-taking in the human brain: An activation likelihood estimation meta-analysis of the balloon analog risk task (BART)

Functionally similar yet distinct neural mechanisms underlie different choice behaviors: ALE meta-analyses of decision-making under risk in adolescents and adults

Neurophysiological insights into sequential decision-making: exploring the secretary problem through ERPs and TBR dynamics

Humans rationally balance detailed and temporally abstract world models

Neural basis of concurrent deliberation toward a choice and degree of confidence

The assessment of affective decision‐making: Exploring alternative scoring methods for the Balloon Analog Risk Task and Columbia Card Task

Description of Sequential Risky Decision-Making Choices in Human-Machine Teams Using Eye-Tracking and Decision Tree.

Brain Mechanisms for Perceptual and Reward-Related Decision-Making