Abstract:Learning to make adaptive decisions involves making choices, assessing their consequence, and leveraging this assessment to attain higher rewarding states. Despite vast literature on value-based decision-making, relatively little is known about the cognitive processes underlying decisions in highly uncertain contexts. Real world decisions are rarely accompanied by immediate feedback, explicit rewards, or complete knowledge of the environment. Being able to make informed decisions in such contexts requires significant knowledge about the environment, which can only be gained via exploration. Here we aim at understanding and formalizing the brain mechanisms underlying these processes. To this end, we first designed and performed an experimental task. Human participants had to learn to maximize reward while making sequences of decisions with only basic knowledge of the environment, and in the absence of explicit performance cues. Participants had to rely on their own internal assessment of performance to reveal a covert relationship between their choices and their subsequent consequences to find a strategy leading to the highest cumulative reward. Our results show that the participants' reaction times were longer whenever the decision involved a future consequence, suggesting greater introspection whenever a delayed value had to be considered. The learning time varied significantly across participants. Second, we formalized the neurocognitive processes underlying decision-making within this task, combining mean-field representations of competing neural populations with a reinforcement learning mechanism. This model provided a plausible characterization of the brain dynamics underlying these processes, and reproduced each aspect of the participants' behavior, from their reaction times and choices to their learning rates. In summary, both the experimental results and the model provide a principled explanation to how delayed value may be computed and incorporated into the neural dynamics of decision-making, and to how learning occurs in these uncertain scenarios.

Decision-making and Fuzzy Temporal Logic

Attribute Latencies Causally Shape Intertemporal Decisions

A formal analysis of inconsistent decisions in intertemporal choice through subjective time perception

A general theory of intertemporal decision-making and the perception of time

Fuzziness and Uncertainty in Temporal Reasoning

Temporal Fairness in Decision Making Problems

Dynamic Decision Making of Decision-Makers' Psychological Expectations Based on Interval Triangular Fuzzy Soft Sets

The additive property of the inconsistency degree in intertemporal decision making through the generalization of psychophysical laws

Use of Electroencephalography for the Study of Gain–Loss Asymmetry in Intertemporal Decision-Making

The neural substrates of probabilistic and intertemporal decision making

A Decision Method for Linear Temporal Logic

Fuzzy Markov Decision-Making Model for Interference Effects

The relationship between temporal discounting and foraging

Critical tests of fuzzy trace theory in brain and behavior: uncertainty across time, probability, and development

Revisiting Fuzzy and Linguistic Decision Making: Scenarios and Challenges for Making Wiser Decisions in a Better Way

A Time-Preference and VIKOR-based Dynamic Intuitionistic Fuzzy Decision Making Method

Is a bird in the hand worth two in the future? The neuroeconomics of intertemporal decision-making

The role of time estimation in decreased impatience in intertemporal choice.

A Hesitant Fuzzy Linguistic Bidirectional Projection-Regret decision making model

Cognitive mechanisms of learning in sequential decision-making under uncertainty: an experimental and theoretical approach

Now for sure or later with a risk? Modeling risky intertemporal choice as accumulated preference.