A short note on basic of information geometry from thermodynamics and thermostatistics

Tatsuaki Wada

DOI: https://doi.org/10.48550/arXiv.2109.02438

2021-09-01

Statistical Mechanics

Abstract:This is a short note for some basics of information geometry from thermodynamics and Callen's themostatistics.

What problem does this paper attempt to address?

The Generalized Thermodynamic Temperature and the New Expressions of the First and the Second Law of Thermodynamics

Yacai Hu,Qi Chen,Yayuan Hu

DOI: https://doi.org/10.1007/s11630-016-0827-1

IF: 2.013

2016-01-01

Journal of Thermal Science

Abstract:The classical thermodynamics reflects the significant relationship between the heat and the temperature. On the basis of the relationships, according to the mathematical derivation, this paper structures the conceptions of generalized heat, generalized thermodynamic temperature, generalized entropy and so on. The series of conceptions in the classical thermodynamics is merely a special case of the generalized thermodynamics. Based on these conceptions of generalized thermodynamics, this paper presents the new expressions of the first law and the second law of thermodynamics. In other words, these expressions are endued with new explanations. The Eq. LZ = kTS given by this paper provides theoretical basis for these new expressions.
Information Geometry for the Working Information Theorist

Kumar Vijay Mishra,M. Ashok Kumar,Ting-Kam Leonard Wong

2023-10-06

Abstract:Information geometry is a study of statistical manifolds, that is, spaces of probability distributions from a geometric perspective. Its classical information-theoretic applications relate to statistical concepts such as Fisher information, sufficient statistics, and efficient estimators. Today, information geometry has emerged as an interdisciplinary field that finds applications in diverse areas such as radar sensing, array signal processing, quantum physics, deep learning, and optimal transport. This article presents an overview of essential information geometry to initiate an information theorist, who may be unfamiliar with this exciting area of research. We explain the concepts of divergences on statistical manifolds, generalized notions of distances, orthogonality, and geodesics, thereby paving the way for concrete applications and novel theoretical investigations. We also highlight some recent information-geometric developments, which are of interest to the broader information theory community.

Information Theory,Machine Learning,Signal Processing,Differential Geometry
On Thermodynamic Information

Bing Miao,Hong Qian,Yong-Shi Wu

2023-12-06

Abstract:Information based thermodynamic logic is revisited. It consists of two parts: Part A applies the modern theory of probability in which an arbitrary convex function \phi is employed as an analytic "device" to express information as statistical dependency contained in the topological sub-\sigma-algebra structure. Via thermo-doubling, Fenchel-Young equality (FYE) that consists of \phi(x) and its conjugate \psi(y) establishes the notion of equilibrium between x and y through duality symmetry and the principle of maximum entropy/minimum free energy. Part B deals with a given set of repetitive measurements, where an inherent convex function emerges via the mathematics of large deviations. Logarithm-based Shannon entropy with \phi(x)=-\log x figures prominently for i.i.d. sample statistics. Information can be a measure of the agreement between a statistical observation and its theoretical models. Maximum likelihood principle arises here and FYE provides a thermodynamic energetic narrative of recurrent data.

Statistical Mechanics,Mathematical Physics
Mechanics of geodesics in Information geometry and Black Hole Thermodynamics

Sumanto Chanda,Tatsuaki Wada

DOI: https://doi.org/10.1142/S0219887824500981

2023-12-06

Abstract:In this article we shall discuss the theory of geodesics in information geometry, and an application in astrophysics. We will study how gradient flows in information geometry describe geodesics, explore the related mechanics by introducing a constraint, and apply our theory to Gaussian model and black hole thermodynamics. Thus, we demonstrate how deformation of gradient flows leads to more general Randers-Finsler metrics, describe Hamiltonian mechanics that derive from a constraint, and prove duality via canonical transformation. We also verified our theories for a deformation of the Gaussian model, and described dynamical evolution of flat metrics for Kerr and Reissner-Nordström black holes.

Information Theory
Thermodynamics of Information

Juan M. R. Parrondo

2023-06-21

Abstract:As early as 1867, two years after the introduction of the concept of entropy by Clausius, Maxwell showed that the limitations imposed by the second law of thermodynamics depend on the information that one possesses about the state of a physical system. A "very observant and neat-fingered being", later on named Maxwell demon by Kelvin, could arrange the molecules of a gas and induce a temperature or pressure gradient without performing work, in apparent contradiction to the second law. One century later, Landauer claimed that "information is physical", and showed that certain processes involving information, like overwriting a memory, need work to be completed and are unavoidably accompanied by heat dissipation. Thermodynamics of information analyzes this bidirectional influence between thermodynamics and information processing. The seminal ideas that Landauer and Bennett devised in the 1970s have been recently reformulated in a more precise and general way by realizing that informational states are out of equilibrium and applying new tools from non-equilibrium statistical mechanics.

Statistical Mechanics,History and Philosophy of Physics
Interrelatedness of thermodynamics and information: transformation of heat as a measurable information process and quantity, an overview

B. Hejna,J. Šesták,Hejna, B.,Šesták, J.

DOI: https://doi.org/10.1007/s10973-023-12833-7

IF: 4.755

2024-03-05

Journal of Thermal Analysis and Calorimetry

Abstract:No one has yet investigated the information reach and influence of thermal analysis. Even so the heat and entropy are bound closely. The very problem of the term entropy is that it was defined specifically within the classical, equilibrium Thermodynamics like Clausius entropy, while its meaning is more general. It is usable in all areas of physics, in the communication theory and technology, and also, in social sciences, just as an universal quantitative model. The bridge to this wider world is in Boltzmann's statistical point of view used in the Shannon's information point of view. The difference is that the Shannon entropy is defined more generally, both for the uniform and for the not uniform probability distributions. But with renormalization and with the Boltzmann constant we can speak, using the Clausius entropy, about a certain thermodynamic system considered as a heat model of the area of our wider interests. So we will to deal with the heat and its entropy embodied to a wider consideration. In addition to the standard physical meaning, another thermodynamic realization in the form of a message is derived from Shannon's information theory, where the sample carries a measure of Shannon's information given by its structure or internal organization. This is also a contribution to improving the approach to thinking about the importance of observation and measurement, reflecting at the same time the costs and influence of the measurement itself and its organization on the measured object or, on its environment or, even on the environment of the measuring itself generally. Statistical sorting also carries with it the internal movement of individual elementary particles, the value of which indicates the resulting state of heat transfer and dissipation to and from the system providing a state of called equilibrium. It is statistically described by a weighted quantity called temperature, the changes of which via momentum alteration satisfy Newton's laws of inertia allowing enumeration of the related heat information processing. The terminology used is still non-institutional.

chemistry, physical, analytical,thermodynamics
Information geometry approach to quantum stochastic thermodynamics

Laetitia P. Bettmann,John Goold

2024-09-20

Abstract:Recent advancements have revealed new links between information geometry and classical stochastic thermodynamics, particularly through the Fisher information with respect to time. Recognizing the non-uniqueness of the quantum Fisher metric in Hilbert space, we exploit the fact that any quantum Fisher information (QFI) can be decomposed into a metric-independent incoherent part and a metric-dependent coherent contribution. We demonstrate that the incoherent component of any QFI can be directly linked to entropic acceleration, and for GKSL dynamics with local detailed balance, to the rate of change of generalized thermodynamic forces and entropic flow, paralleling the classical results. Furthermore, we show that the classical uncertainty relation between the geometric uncertainty of a path in state space and the time-averaged rate of information change also holds for quantum systems. We generalise a classical geometric bound on the entropy rate for far-from-equilibrium processes by incorporating a non-negative quantum contribution that arises from the geometric action due to coherent dynamics. Finally, we apply an information-geometric analysis to the recently proposed quantum-thermodynamic Mpemba effect, demonstrating this framework's ability to capture thermodynamic phenomena.

Quantum Physics,Statistical Mechanics
Axiomatic Relation between Thermodynamic and Information-Theoretic Entropies

Mirjam Weilenmann,Lea Kraemer,Philippe Faist,Renato Renner

DOI: https://doi.org/10.1103/physrevlett.117.260601

IF: 8.6

2016-12-22

Physical Review Letters

Abstract:Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a measure of uncertainty. In this Letter, we connect these two notions of entropy, using an axiomatic framework for thermodynamics [E. H. Lieb and J. Yngvason Proc. R. Soc. 469, 20130408 (2013)]. In particular, we obtain a direct relation between the Clausius entropy and the Shannon entropy, or its generalization to quantum systems, the von Neumann entropy. More generally, we find that entropy measures relevant in nonequilibrium thermodynamics correspond to entropies used in one-shot information theory.

physics, multidisciplinary
The basics of information geometry

Ariel Caticha

DOI: https://doi.org/10.1063/1.4905960

2015-01-01

AIP Conference Proceedings

Abstract:To what extent can we distinguish one probability distribution from another? Are there quantitative measures of distinguishability? The goal of this tutorial is to approach such questions by introducing the notion of the “distance” between two probability distributions and exploring some basic ideas of such an “information geometry”.
Affine geometric description of thermodynamics

Shin-itiro Goto

DOI: https://doi.org/10.1063/5.0124768

2023-03-10

Abstract:Thermodynamics provides a unified perspective of thermodynamic properties of various substances. To formulate thermodynamics in the language of sophisticated mathematics, thermodynamics is described by a variety of differential geometries, including contact and symplectic geometries. Meanwhile affine geometry is a branch of differential geometry and is compatible with information geometry, where information geometry is known to be compatible with thermodynamics. By combining above, it is expected that thermodynamics is compatible with affine geometry, and is expected that several affine geometric tools can be introduced in the analysis of thermodynamic systems. In this paper affine geometric descriptions of equilibrium and nonequilibrium thermodynamics are proposed. For equilibrium systems, it is shown that several thermodynamic quantities can be identified with geometric objects in affine geometry, and that several geometric objects can be introduced in thermodynamics. Examples of these include: specific heat is identified with the affine fundamental form, a flat connection is introduced in thermodynamic phase space. For nonequilibrium systems, two classes of relaxation processes are shown to be described in the language of an extension of affine geometry. Finally this affine geometric description of thermodynamics for equilibrium and nonequilibrium systems is compared with a contact geometric description.

Mathematical Physics
Nonequilibrium thermodynamics and information theory: Basic concepts and relaxing dynamics

Bernhard Altaner

DOI: https://doi.org/10.1088/1751-8121/aa841d

2017-07-06

Abstract:Thermodynamics is based on the notions of energy and entropy. While energy is the elementary quantity governing physical dynamics, entropy is the fundamental concept in information theory. In this work, starting from first principles, we give a detailed didactic account on the relations between energy and entropy and thus physics and information theory. We show that thermodynamic process inequalities, like the Second Law, are equivalent to the requirement that an effective description for physical dynamics is strongly relaxing. From the perspective of information theory, strongly relaxing dynamics govern the irreversible convergence of a statistical ensemble towards the maximally non-commital probability distribution that is compatible with thermodynamic equilibrium parameters. In particular, Markov processes that converge to a thermodynamic equilibrium state are strongly relaxing. Our framework generalizes previous results to arbitrary open and driven systems, yielding novel thermodynamic bounds for idealized and real processes.

Statistical Mechanics
Nonequilibrium in Thermodynamic Formalism: the Second Law, gases and Information Geometry

Artur O. Lopes,R. Ruggiero

DOI: https://doi.org/10.48550/arXiv.2103.08333

2021-11-24

Abstract:In Nonequilibrium Thermodynamics and Information Theory, the relative entropy (or, KL divergence) plays a very important role. Consider a Hölder Jacobian $J$ and the Ruelle (transfer) operator $\mathcal{L}_{\log J}.$ Two equilibrium probabilities $\mu_1$ and $\mu_2$, can interact via a discrete-time {\it Thermodynamic Operation} described by the action {\it of the dual of the Ruelle operator} $ \mathcal{L}_{\log J}^*$. We argue that the law $\mu \to \mathcal{L}_{\log J}^*(\mu)$, producing nonequilibrium, can be seen as a Thermodynamic Operation after showing that it's a manifestation of the Second Law of Thermodynamics. We also show that the change of relative entropy satisfies $$ D_{K L} (\mu_1,\mu_2) - D_{K L} (\mathcal{L}_{\log J}^*(\mu_1),\mathcal{L}_{\log J}^*(\mu_2))= 0.$$ Furthermore, we describe sufficient conditions on $J,\mu_1$ for getting $h(\mathcal{L}_{\log J}^*(\mu_1))\geq h(\mu_1)$, where $h$ is entropy. Recalling a natural Riemannian metric in the Banach manifold of Hölder equilibrium probabilities we exhibit the second-order Taylor formula for an infinitesimal tangent change of KL divergence; a crucial estimate in Information Geometry. We introduce concepts like heat, work, volume, pressure, and internal energy, which play here the role of the analogous ones in Thermodynamics of gases. We briefly describe the MaxEnt method.

Dynamical Systems,Statistical Mechanics,Information Theory,Mathematical Physics,Probability
Thermodynamics Basic Principles

G. V. Skornyakov

DOI: https://doi.org/10.48550/arXiv.1201.4133

2013-03-01

Abstract:Logical and mathematical aspects of the basic concepts of thermodynamics are considered.

General Physics
The Geometry of Some Thermodynamic Systems

Alexandre Anahory Simoes,David Martín de Diego,Manuel Lainz Valcázar,Manuel de León

DOI: https://doi.org/10.1007/978-3-030-77957-3_13

2021-01-01

Abstract:In this article, we continue the program started in [2] of exploring an important class of thermodynamic systems from a geometric point of view. The contents of this paper and the one already published in [2] provide a geometrical formulation, which tries to shed more light on the properties of thermodynamic systems without claiming to be a definitive theory. In order to model the time evolution of systems verifying the two laws of thermodynamics, we show that the notion of evolution vector field is adequate to appropriately describe such systems. Our formulation naturally arises from the introduction of a skew-symmetric bracket to which numerical methods based on discrete gradients fit nicely. Moreover, we study the corresponding Lagrangian and Hamiltonian formalism, discussing the fundamental principles from which the equations are derived. An important class of systems that is naturally covered by our formalism are composed thermodynamic systems, which are described by at least two thermal variables and exchange heat between its components.
Mathematical And Information-Geometrical Entropy For Phenomenological Fourier And Non-Fourier Heat Conduction

Shu-Nan Li,Bing-Yang Cao

DOI: https://doi.org/10.1103/PhysRevE.96.032131

IF: 2.707

2017-01-01

Physical Review E

Abstract:The second law of thermodynamics governs the direction of heat transport, which provides the foundational definition of thermodynamic Clausius entropy. The definitions of entropy are further generalized for the phenomenological heat transport models in the frameworks of classical irreversible thermodynamics and extended irreversible thermodynamics (EIT). In this work, entropic functions from mathematics are combined with phenomenological heat conduction models and connected to several information-geometrical conceptions. The long-time behaviors of these mathematical entropies exhibit a wide diversity and physical pictures in phenomenological heat conductions, including the tendency to thermal equilibrium, and exponential decay of nonequilibrium and asymptotics, which build a bridge between the macroscopic and microscopic modelings. In contrast with the EIT entropies, the mathematical entropies expressed in terms of the internal energy function can avoid singularity paired with nonpositive local absolute temperature caused by non-Fourier heat conduction models.
Reformulation of Classical Thermodynamics from Information Theory

Faical Barzi,Kaoutar Fethi

2024-10-08

Abstract:In this study, we present a reformulation of classical equilibrium thermodynamics by replacing the obscure and ambiguous concept of entropy with the clear and intuitive concept of information stored in a thermodynamic system. Specifically, we rewrite the laws of thermodynamics in the mathematical terminology borrowed from information theory with an emphasis on information instead of entropy and on binary logarithm instead of natural one. We also define a modified ideal gas constant denoted as $\mathcal{R}$ that quantifies the energy cost of storing or retrieving a mole of information. Moreover an application to the ideal gas is carried and new insight on the evolution of the system is acquired. This new formulation might serve as a basis to teaching thermodynamics, where one deals with the concept of energy and information both stored in a thermodynamic system and avoids the lack of cognitive representations inherent in the concept of entropy.

Statistical Mechanics,Classical Physics,Physics Education
Divergence, Entropy, Information: An Opinionated Introduction to Information Theory

Philip Chodrow

DOI: https://doi.org/10.48550/arXiv.1708.07459

2019-03-24

Abstract:Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with relatively little mathematical sophistication, while many others develop specific application areas in detail. In contrast, these informal notes aim to outline some elements of the information-theoretic "way of thinking," by cutting a rapid and interesting path through some of the theory's foundational concepts and results. They are aimed at practicing systems scientists who are interested in exploring potential connections between information theory and their own fields. The main mathematical prerequisite for the notes is comfort with elementary probability, including sample spaces, conditioning, and expectations. We take the Kullback-Leibler divergence as our most basic concept, and then proceed to develop the entropy and mutual information. We discuss some of the main results, including the Chernoff bounds as a characterization of the divergence; Gibbs' Theorem; and the Data Processing Inequality. A recurring theme is that the definitions of information theory support natural theorems that sound ``obvious'' when translated into English. More pithily, ``information theory makes common sense precise.'' Since the focus of the notes is not primarily on technical details, proofs are provided only where the relevant techniques are illustrative of broader themes. Otherwise, proofs and intriguing tangents are referenced in liberally-sprinkled footnotes. The notes close with a highly nonexhaustive list of references to resources and other perspectives on the field.

Information Theory,Statistics Theory,Data Analysis, Statistics and Probability
Geometric thermodynamics for the Fokker-Planck equation: Stochastic thermodynamic links between information geometry and optimal transport

Sosuke Ito

DOI: https://doi.org/10.1007/s41884-023-00102-3

2023-12-13

Abstract:We propose a geometric theory of non-equilibrium thermodynamics, namely geometric thermodynamics, using our recent developments of differential-geometric aspects of entropy production rate in non-equilibrium thermodynamics. By revisiting our recent results on geometrical aspects of entropy production rate in stochastic thermodynamics for the Fokker-Planck equation, we introduce a geometric framework of non-equilibrium thermodynamics in terms of information geometry and optimal transport theory. We show that the proposed geometric framework is useful for obtaining several non-equilibrium thermodynamic relations, such as thermodynamic trade-off relations between the thermodynamic cost and the fluctuation of the observable, optimal protocols for the minimum thermodynamic cost and the decomposition of the entropy production rate for the non-equilibrium system. We clarify several stochastic-thermodynamic links between information geometry and optimal transport theory via the excess entropy production rate based on a relation between the gradient flow expression and information geometry in the space of probability densities and a relation between the velocity field in optimal transport and information geometry in the space of path probability densities.

Statistical Mechanics,Mathematical Physics
Universal bounds on the performance of information-thermodynamic engine

Tomohiro Tanogami,Tan Van Vu,Keiji Saito

DOI: https://doi.org/10.1103/PhysRevResearch.5.043280

2023-10-17

Abstract:We investigate fundamental limits on the performance of information processing systems from the perspective of information thermodynamics. We first extend the thermodynamic uncertainty relation (TUR) to a subsystem. Specifically, for a bipartite composite system consisting of a system of interest X and an auxiliary system Y, we show that the relative fluctuation of an arbitrary current for X is lower bounded not only by the entropy production associated with X but also by the information flow between X and Y. As a direct consequence of this bipartite TUR, we prove universal trade-off relations between the output power and efficiency of an information-thermodynamic engine in the fast relaxation limit of the auxiliary system. In this limit, we further show that the Gallavotti-Cohen symmetry is satisfied even in the presence of information flow. This symmetry leads to universal relations between the fluctuations of information flow and entropy production in the linear response regime. We illustrate these results with simple examples: coupled quantum dots and coupled linear overdamped Langevin equations. Interestingly, in the latter case, the equality of the bipartite TUR is achieved even far from equilibrium, which is a very different property from the standard TUR. Our results will be applicable to a wide range of systems, including biological systems, and thus provide insight into the design principles of biological systems.

Statistical Mechanics
Thermodynamic efficiency of information and heat flow

Armen E Allahverdyan,Dominik Janzing,Guenter Mahler

DOI: https://doi.org/10.1088/1742-5468/2009/09/p09011

2009-09-21

Abstract:A basic task of information processing is information transfer (flow). Here we study a pair of Brownian particles each coupled to a thermal bath at temperatures T1 and T2. The information flow in such a system is defined via the time-shifted mutual information. The information flow nullifies at equilibrium, and its efficiency is defined as the ratio of the flow to the total entropy production in the system. For a stationary state the information flows from higher to lower temperatures, and its efficiency is bounded from above by (max[T1,T2])/(|T1−T2|). This upper bound is imposed by the second law and it quantifies the thermodynamic cost for information flow in the present class of systems. It can be reached in the adiabatic situation, where the particles have widely different characteristic times. The efficiency of heat flow—defined as the heat flow over the total amount of dissipated heat—is limited from above by the same factor. There is a complementarity between heat and information flow: the set-up which is most efficient for the former is the least efficient for the latter and vice versa. The above bound for the efficiency can be (transiently) overcome in certain non-stationary situations, but the efficiency is still limited from above. We study yet another measure of information processing (transfer entropy) proposed in the literature. Though this measure does not require any thermodynamic cost, the information flow and transfer entropy are shown to be intimately related for stationary states.

physics, mathematical,mechanics

A short note on basic of information geometry from thermodynamics and thermostatistics

The Generalized Thermodynamic Temperature and the New Expressions of the First and the Second Law of Thermodynamics

Information Geometry for the Working Information Theorist

On Thermodynamic Information

Mechanics of geodesics in Information geometry and Black Hole Thermodynamics

Thermodynamics of Information

Interrelatedness of thermodynamics and information: transformation of heat as a measurable information process and quantity, an overview

Information geometry approach to quantum stochastic thermodynamics

Axiomatic Relation between Thermodynamic and Information-Theoretic Entropies

The basics of information geometry

Affine geometric description of thermodynamics

Nonequilibrium thermodynamics and information theory: Basic concepts and relaxing dynamics

Nonequilibrium in Thermodynamic Formalism: the Second Law, gases and Information Geometry

Thermodynamics Basic Principles

The Geometry of Some Thermodynamic Systems

Mathematical And Information-Geometrical Entropy For Phenomenological Fourier And Non-Fourier Heat Conduction

Reformulation of Classical Thermodynamics from Information Theory

Divergence, Entropy, Information: An Opinionated Introduction to Information Theory

Geometric thermodynamics for the Fokker-Planck equation: Stochastic thermodynamic links between information geometry and optimal transport

Universal bounds on the performance of information-thermodynamic engine

Thermodynamic efficiency of information and heat flow