Information thermodynamics: from physics to neuroscience

Jan Karbowski

DOI: https://doi.org/10.3390/e26090779

2024-09-26

Abstract:This paper provides a perspective on applying the concepts of information thermodynamics, developed recently in non-equilibrium statistical physics, to problems in theoretical neuroscience. Historically, information and energy in neuroscience have been treated separately, in contrast to physics approaches, where the relationship of entropy production with heat is a central idea. It is argued here that also in neural systems information and energy can be considered within the same theoretical framework. Starting from basic ideas of thermodynamics and information theory on a classic Brownian particle, it is shown how noisy neural networks can infer its probabilistic motion. The decoding of the particle motion by neurons is performed with some accuracy and it has some energy cost, and both can be determined using information thermodynamics. In a similar fashion, we also discuss how neural networks in the brain can learn the particle velocity, and maintain that information in the weights of plastic synapses from a physical point of view. Generally, it is shown how the framework of stochastic and information thermodynamics can be used practically to study neural inference, learning, and information storing.

Neurons and Cognition,Disordered Systems and Neural Networks,Statistical Mechanics,Biological Physics

What problem does this paper attempt to address?

The core issue this paper attempts to address is the application of the concepts of information thermodynamics to theoretical neuroscience, in order to establish a unified framework that simultaneously considers information and energy in neural systems. Specifically, the paper explores how the concepts of information thermodynamics developed in non-equilibrium statistical physics can be used to understand the processes of information processing, learning, and storage in neural networks. Traditional neuroscience research typically treats information and energy separately, whereas this paper proposes that, just as the relationship between entropy production and heat is a core concept in physics, information and energy in neural systems can also be studied within a unified theoretical framework.

By analyzing the classic example of Brownian particles, the paper demonstrates how the methods of information thermodynamics can be used to study how noisy neural networks infer the probabilistic motion of particles and the associated energy costs. Additionally, it discusses how neural networks in the brain learn the velocity of particles from a physical perspective and store this information in the weights of plastic synapses. Overall, the paper shows how practical frameworks of stochastic and information thermodynamics can be used to study neural inference, learning, and information storage.

Through these studies, the paper aims to provide new perspectives for understanding the efficient information processing mechanisms of neural systems and emphasizes that energy is an important constraint in the process of neural information processing and storage. This not only aids in theoretical understanding but may also offer insights for designing more efficient computational models and algorithms.