Learning and embodied decisions in active inference

Matteo Priorelli,Ivilin Peev Stoianov,Giovanni Pezzulo
DOI: https://doi.org/10.1101/2024.08.18.608439
2024-08-21
Abstract:Biological organisms constantly face the necessity to act timely in dynamic environments and balance choice accuracy against the risk of missing valid opportunities. As formalized by embodied decision models, this might require brain architectures wherein decision-making and motor control interact reciprocally, in stark contrast to traditional models that view them as serial processes. Previous studies have assessed that embodied decision dynamics emerge naturally under active inference - a computational paradigm that considers action and perception as subject to the same imperative of free energy minimization. In particular, agents can infer their targets by using their own movements (and not only external sensations) as evidence, i.e., via self-evidencing. Such models have shown that under appropriate conditions, action-generated feedback can stabilize and improve decision processes. However, how adaptation of internal models to environmental contingencies influences embodied decisions is yet to be addressed. To shed light on this challenge, in this study we systematically investigate the learning dynamics of an embodied model of decision-making during a two-alternative forced choice task, using a hybrid (discrete and continuous) active inference framework. Our results show that active inference agents can adapt to embodied contexts by learning various statistical regularities of the task - namely, prior preferences for the correct target, cue validity, and response strategies that prioritize faster or slower (but more accurate) decisions. Crucially, these results illustrate the efficacy of learning discrete preferences and strategies using sensorimotor feedback from continuous dynamics.
Neuroscience
What problem does this paper attempt to address?
### Problems the Paper Aims to Solve This paper aims to explore how organisms make timely decisions through the interaction of action and perception in dynamic environments, balancing the accuracy of choices with the risk of missing effective opportunities. Specifically, the paper investigates how decision models based on the active inference framework learn statistical regularities in the environment in a two-alternative forced choice task. The main focuses include: 1. **Prior Learning of Correct Targets**: Investigating how the model learns the distribution of correct targets under different conditions, especially in reversal learning tasks. 2. **Learning of Cue Validity**: Exploring how the model adjusts its estimation of cue validity according to environmental changes. 3. **Learning of Response Strategies**: Analyzing how the model balances between making fast and accurate decisions. Through these studies, the paper demonstrates that active inference models can adapt to different decision-making contexts and optimize discrete preferences and strategies through continuous sensorimotor feedback.