Abstract:Learning to make adaptive decisions involves making choices, assessing their consequence, and leveraging this assessment to attain higher rewarding states. Despite vast literature on value-based decision-making, relatively little is known about the cognitive processes underlying decisions in highly uncertain contexts. Real world decisions are rarely accompanied by immediate feedback, explicit rewards, or complete knowledge of the environment. Being able to make informed decisions in such contexts requires significant knowledge about the environment, which can only be gained via exploration. Here we aim at understanding and formalizing the brain mechanisms underlying these processes. To this end, we first designed and performed an experimental task. Human participants had to learn to maximize reward while making sequences of decisions with only basic knowledge of the environment, and in the absence of explicit performance cues. Participants had to rely on their own internal assessment of performance to reveal a covert relationship between their choices and their subsequent consequences to find a strategy leading to the highest cumulative reward. Our results show that the participants' reaction times were longer whenever the decision involved a future consequence, suggesting greater introspection whenever a delayed value had to be considered. The learning time varied significantly across participants. Second, we formalized the neurocognitive processes underlying decision-making within this task, combining mean-field representations of competing neural populations with a reinforcement learning mechanism. This model provided a plausible characterization of the brain dynamics underlying these processes, and reproduced each aspect of the participants' behavior, from their reaction times and choices to their learning rates. In summary, both the experimental results and the model provide a principled explanation to how delayed value may be computed and incorporated into the neural dynamics of decision-making, and to how learning occurs in these uncertain scenarios.

What's in a sample? Epistemic uncertainty and metacognitive awareness in risk taking

Gambling under unknown probabilities as a proxy for real world decisions under uncertainty

Rational Decision-Making Under Uncertainty: Observed Betting Patterns on a Biased Coin

"Sometimes You Just Gotta Risk It for the Biscuit": A Portrait of Student Risk-Taking

Cognitive processes underlying lottery and sports gambling decisions: The role of stated probabilities and background knowledge

Understanding the implications of low knowledge and high uncertainty in risk studies

Surprising sounds influence risky decision making

Gambling in risk-taking contests: Experimental evidence

A Set of Estimation and Decision Preference Experiments for Exploring Risk Assessment Biases in Engineering Students

Cognitive mechanisms of learning in sequential decision-making under uncertainty: an experimental and theoretical approach

Adaptive Integration of Perceptual and Reward Information in an Uncertain World

A Probabilistic Approach to Epistemic Safety from the Perspective of Ascribers

Are decisions under risk malleable?

Rapid Changes in Risk Preferences Originate from Bayesian Inference on Parietal Magnitude Representations

Investment Decisions and Time Horizon: Risk Perception and Risk Behavior in Repeated Gambles

Further perceptions of probability: Accurate, stepwise updating is contingent on prior information about the task and the response mode

Epistemic Risk-Sensitive Reinforcement Learning

Confidence control for efficient behaviour in dynamic environments

Behavioral Biases and Nonadditive Dynamics in Risk Taking: An Experimental Investigation

Distinguishing Risk Preferences using Repeated Gambles

Distinct computational mechanisms of uncertainty processing explain opposing exploratory behaviors in anxiety and apathy