Abstract:The relationship between time irreversibility in neuronal dynamics and cognitive effort is a subject of growing interest in the scientific literature. Although correlations between proxies of both concepts have been experimentally observed, the underlying precise linkage between them remains elusive. Here we investigate the case of learning in decision-making tasks; we do so by introducing a thermodynamically grounded metric-inspired by Landauer's principle-which connects time-irreversible information processing to energy consumption. Equipped with this metric, we investigate the role of macroscopic time-reversal symmetry breaking in belief dynamics for the case of an agent with finite sensitivity while performing a static two-armed bandit task-a standard setup in cognitive neuroscience. To gain insights into the belief dynamics, we analogize it to the dynamics of an active particle subject to state-dependent noise and living in a two-dimensional space. This mapping allows an analytical description of learning-induced biases. We deeply explore the case of Q-learning with forgetting the nonchosen option. In this case, learning-induced risk aversion is formally equivalent to standard thermophoresis, i.e., the net motion towards low-temperature regions. Finally, we quantify the irreversibility of belief dynamics in the steady state for different bandit configurations, sensitivity levels, and exploitative behavior. We found a strong correlation in high-sensitivity learning between heightened irreversibility in belief dynamics and improved decision-making outcomes. Notably, as the task's difficulty increases, a greater degree of irreversibility in belief dynamics becomes necessary for having superior performances; this explicitly unravels a plausible connection between time irreversibility and cognitive effort. In conclusion, our investigation reveals that irreversibility in belief dynamics bridges out-of-equilibrium statistical physics concepts and cognitive neuroscience. In decision-making contexts, this perspective offers insights into the notion of cognitive effort, suggesting a potential mechanism driving the evolution of living systems toward out-of-equilibrium structures.

Identifying Assumptions and Research Dynamics

Identifying Latent Dynamic Components in Biological Systems

Modeling Information Acquisition and Social Learning Dynamics: A Rational Inattention Perspective

Learning and choosing in an uncertain world: An investigation of the explore-exploit dilemma in static and dynamic environments

Belief dynamics extraction

Instance-based learning: Integrating sampling and repeated decisions from experience.

Planned behavior, perceptual biases, and the dynamics of collective action

Asymptotic Behavior of Bayesian Learners with Misspecified Models

Life Cycle and the Behavior of Research

A Framework for Eliciting, Incorporating, and Disciplining Identification Beliefs in Linear Models

Social setting, intuition, and experience in lab experiments interact to shape cooperative decision-making

Logical Dynamics of Belief Change in the Community.

Prespecification of Structure for Optimizing Data Collection and Research Transparency by Leveraging Conditional Independencies

Beyond Occam's Razor in System Identification: Double-Descent when Modeling Dynamics

Time-evolving psychological processes over repeated decisions

Further perceptions of probability: Accurate, stepwise updating is contingent on prior information about the task and the response mode

Doing Experiments and Revising Rules with Natural Language and Probabilistic Reasoning

Irreversibility in belief dynamics: Unraveling the link to cognitive effort

Humans rationally balance detailed and temporally abstract world models

Non-strategic Econometrics (for Initial Play)

The Tweedledum and Tweedledee of dynamic decisions: Discriminating between diffusion decision and accumulator models