Abstract:This article reviews how organisms learn and recognize the world through the dynamics of neural networks from the perspective of Bayesian inference, and introduces a view on how such dynamics is described by the laws for the entropy of neural activity, a paradigm that we call thermodynamics of the Bayesian brain. The Bayesian brain hypothesis sees the stimulus-evoked activity of neurons as an act of constructing the Bayesian posterior distribution based on the generative model of the external world that an organism possesses. A closer look at the stimulus-evoked activity at early sensory cortices reveals that feedforward connections initially mediate the stimulus-response, which is later modulated by input from recurrent connections. Importantly, not the initial response, but the delayed modulation expresses animals' cognitive states such as awareness and attention regarding the stimulus. Using a simple generative model made of a spiking neural population, we reproduce the stimulus-evoked dynamics with the delayed feedback modulation as the process of the Bayesian inference that integrates the stimulus evidence and a prior knowledge with time-delay. We then introduce a thermodynamic view on this process based on the laws for the entropy of neural activity. This view elucidates that the process of the Bayesian inference works as the recently-proposed information-theoretic engine (neural engine, an analogue of a heat engine in thermodynamics), which allows us to quantify the perceptual capacity expressed in the delayed modulation in terms of entropy.

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is to understand how organisms learn and perceive the world through the dynamic processes of neural networks. In particular, it interprets these processes from the perspective of Bayesian inference and introduces a thermodynamic perspective based on the entropy of neural activity to describe such dynamic processes. Specifically, the paper explores the following aspects: 1. **Bayesian Brain Hypothesis**: The paper holds that the brain infers the causes leading to sensory inputs by constructing a generative model of the external world. This process can be achieved through the construction of the Bayesian posterior distribution, that is, the brain uses the generative model and Bayes' theorem to infer the causal relationships in the external world. 2. **Dynamic Characteristics of Neural Activity**: The paper points out that when a stimulus acts on the sensory cortex, the initial response is mainly mediated by feed - forward connections, while the subsequent modulation is affected by the input from recurrent connections. Importantly, this delayed modulation reflects the animal's cognitive state towards the stimulus, such as consciousness and attention. 3. **Information - Theoretic Engine (Neural Engine)**: The paper proposes a new paradigm, which regards the Bayesian inference process as an information - theoretic engine, similar to a heat engine in thermodynamics. Through this perspective, the perceptual capacity can be quantified, especially the perceptual ability expressed in the delayed modulation, which is measured by the change of entropy. 4. **Thermodynamic Approach**: The paper also introduces how to study the dynamic processes of learning and cognition from a thermodynamic perspective, especially the model obtained through the maximum entropy principle to describe the entropy conservation law (first law) of neural activity, and the conditions of the entropy increase law (second law) in the learning process. In summary, this paper aims to provide a new understanding of the dynamic processes of neural networks through the unified perspective of Bayesian and thermodynamic, especially in - depth exploration of how organisms learn and perceive through neural activities.

The principles of adaptation in organisms and machines II: Thermodynamics of the Bayesian brain

The principles of adaptation in organisms and machines I: machine learning, information theory, and thermodynamics

Advanced Models of Cortical Dynamics in Perception

Information Thermodynamics: From Physics to Neuroscience

Bayesian Mechanics of Synaptic Learning under the Free Energy Principle

The "Bayesian" brain, with a bit less Bayes

Bayesian mechanics of perceptual inference and motor control in the brain

Probabilistic Principles for Biophysics and Neuroscience: Entropy Production, Bayesian Mechanics & the Free-Energy Principle

Free energy and inference in living systems

Signatures of Bayesian inference emerge from energy efficient synapses

Bayesian mechanics of self-organising systems

Evolutionary emergence of biological intelligence

Recognition Dynamics in the Brain under the Free Energy Principle

Non-equilibrium dynamics of adaptation in sensory systems

Neuronal Sequence Models for Bayesian Online Inference

The brain as a probabilistic transducer: an evolutionarily plausible network architecture for knowledge representation, computation, and behavior

Neural and phenotypic representation under the free-energy principle

The thermodynamic brain and the evolution of intellect: the role of mental energy

A Goal-Driven Approach to Systems Neuroscience

Neural substrate of dynamic Bayesian inference in the cerebral cortex