How and in What Sense Can the Entropy Be Measured?

Bin Zhou

DOI: https://doi.org/10.48550/arXiv.physics/0208052

2002-08-13

General Physics

Abstract:In this paper, a method of measuring the entropy is presented. Problems related to the entropy and the heat are also discussed.

What problem does this paper attempt to address?

Measurement-induced Nonlocality Based on the Relative Entropy

Zhengjun Xi,Xiaoguang Wang,Yongming Li

DOI: https://doi.org/10.1103/physreva.85.042325

2012-01-01

Abstract:We quantify the measurement-induced nonlocality [Luo and Fu, Phys. Rev. Lett. 106, 120401 (2011)] from the perspective of the relative entropy. This quantification leads to an operational interpretation for the measurement-induced nonlocality, namely, it is the maximal entropy increase after the locally invariant measurements. The relative entropy of nonlocality is upper bounded by the entropy of the measured subsystem. We establish a relationship between the relative entropy of nonlocality and the geometric nonlocality based on the Hilbert-Schmidt norm, and show that it is equal to the maximal distillable entanglement. Several trade-off relations are obtained for tripartite pure states. We also give explicit expressions for the relative entropy of nonlocality for Bell-diagonal states.
Dimensional measures of generalized entropy

Vladimir Zhdankin

DOI: https://doi.org/10.1088/1751-8121/acf175

2023-08-18

Abstract:Entropy is useful in statistical problems as a measure of irreversibility, randomness, mixing, dispersion, and number of microstates. However, there remains ambiguity over the precise mathematical formulation of entropy, generalized beyond the additive definition pioneered by Boltzmann, Gibbs, and Shannon (applicable to thermodynamic equilibria). For generalized entropies to be applied rigorously to nonequilibrium statistical mechanics, we suggest that there is a need for a physically interpretable (dimensional) framework that can be connected to dynamical processes operating in phase space. In this work, we introduce dimensional measures of entropy that admit arbitrary invertible weight functions (subject to curvature and convergence requirements). These "dimensional entropies" have physical dimensions of phase-space volume and represent the extent of level sets of the distribution function. Dimensional entropies with power-law weight functions (related to Rényi and Tsallis entropies) are particularly robust, as they do not require any internal dimensional parameters due to their scale invariance. We also point out the existence of composite entropy measures that can be constructed from functionals of dimensional entropies. We calculate the response of the dimensional entropies to perturbations, showing that for a structured distribution, perturbations have the largest impact on entropies weighted at a similar phase-space scale. This elucidates the link between dynamics (perturbations) and statistics (entropies). Finally, we derive corresponding generalized maximum-entropy distributions. Dimensional entropies may be useful as a diagnostic (for irreversibility) and for theoretical modeling (if the underlying irreversible processes in phase space are understood) in chaotic and complex systems, such as collisionless systems of particles with long-range interactions.

Statistical Mechanics,Mathematical Physics,Data Analysis, Statistics and Probability,Plasma Physics
Entropy of Quantum Measurements

Stan Gudder

DOI: https://doi.org/10.3390/e24111686

2022-11-02

Abstract:If $a$ is a quantum effect and $\rho$ is a state, we define the $\rho$-entropy $S_a(\rho )$ which gives the amount of uncertainty that a measurement of $a$ provides about $\rho$. The smaller $S_a(\rho )$ is, the more information a measurement of $a$ gives about $\rho$. In Section~2, we provide bounds on $S_a(\rho )$ and show that if $a+b$ is an effect, then $S_{a+b}(\rho )\ge S_a(\rho )+S_b(\rho )$. We then prove a result concerning convex mixtures of effects. We also consider sequential products of effects and their $\rho$-entropies. In Section~3, we employ $S_a(\rho )$ to define the $\rho$-entropy $S_A(\rho )$ for an observable $A$. We show that $S_A(\rho )$ directly provides the $\rho$-entropy $S_\iscript (\rho )$ for an instrument $\iscript$. We establish bounds for $S_A(\rho )$ and prove characterizations for when these bounds are obtained. These give simplified proofs of results given in the literature. We also consider $\rho$-entropies for measurement models, sequential products of observables and coarse-graining of observables. Various examples that illustrate the theory are provided.

Quantum Physics
An exploration for the macroscopic physical meaning of entropy

Jing Wu,ZengYuan Guo

DOI: https://doi.org/10.1007/s11431-010-3227-x

2010-01-01

Abstract:The macroscopic physical meaning of entropy is analyzed based on the exergy (availability) of a combined system (a closed system and its environment), which is the maximum amount of useful work obtainable from the system and the environment as the system is brought into equilibrium with the environment. The process the system experiences can be divided in two sequent sub-processes, the process at constant volume, which represents the heat interaction of the system with the environment, and the adiabatic process, which represents the work interaction of the system with the environment. It is shown that the macroscopic physical meaning of entropy is a measure of the unavailable energy of a closed system for doing useful work through heat interaction. This statement is more precise than those reported in prior literature. The unavailability function of a closed system can be defined as T 0 S and p 0 V in volume constant process and adiabatic process, respectively. Their changes, that is, Δ( T 0 S ) and Δ( p 0 V ) represent the unusable parts of the internal energy of a closed system for doing useful work in corresponding processes. Finally, the relation between Clausius entropy and Boltzmann entropy is discussed based on the comparison of their expressions for absolute entropy.
Entropy And Its Correlations With Other Related Quantities

Jing Wu,Zeng-Yuan Guo

DOI: https://doi.org/10.3390/e16021089

IF: 2.738

2014-01-01

Entropy

Abstract:In order to find more correlations between entropy and other related quantities, an analogical analysis is conducted between thermal science and other branches of physics. Potential energy in various forms is the product of a conserved extensive quantity ( for example, mass or electric charge) and an intensive quantity which is its potential ( for example, gravitational potential or electrical voltage), while energy in specific form is a dissipative quantity during irreversible transfer process ( for example mechanical or electrical energy will be dissipated as thermal energy). However, it has been shown that heat or thermal energy, like mass or electric charge, is conserved during heat transfer processes. When a heat transfer process is for object heating or cooling, the potential of internal energy U is the temperature T and its potential " energy" is UT/2 ( called entransy and it is the simplified expression of thermomass potential energy); when a heat transfer process is for heat-work conversion, the potential of internal energy U is ( 1 - T-0/ T), and the available potential energy of a system in reversible heat interaction with the environment is U - U-0 - T-0( S - S-0), then T-0/ T and T-0( S - S-0) are the unavailable potential and the unavailable potential energy of a system respectively. Hence, entropy is related to the unavailable potential energy per unit environmental temperature for heat- work conversion during reversible heat interaction between the system and its environment. Entropy transfer, like other forms of potential energy transfer, is the product of the heat and its potential, the reciprocal of temperature, although it is in form of the quotient of the heat and the temperature. Thus, the physical essence of entropy transfer is the unavailable potential energy transfer per unit environmental temperature. Entropy is a non- conserved, extensive, state quantity of a system, and entropy generation in an irreversible heat transfer process is proportional to the destruction of available potential energy.
Measure-theoretic metric mean dimension

Rui Yang,Ercai Chen,Xiaoyao Zhou

2024-08-31

Abstract:For infinite measure-theoretic entropy systems, we introduce the notion of measure-theoretic metric mean dimension of invariant measures for different types of measure-theoretic $\epsilon$-entropies, and show that measure-theoretic metric mean dimensions of different types of measure-theoretic $\epsilon$-entropies coincide with the packing metric mean dimension of the set of generic points of ergodic measures.

Dynamical Systems
Experimental Device-Independent Tests of Classical and Quantum Entropy

Feng Zhu,Wei Zhang,Sijing Chen,Lixing You,Zhen Wang,Yidong Huang

DOI: https://doi.org/10.1103/physreva.94.062340

2016-01-01

Abstract:In quantum information processing, it is important to witness the entropy of the message in the device-independent way which was proposed recently [R. Chaves, J. B. Brask, and N. Brunner, Phys. Rev. Lett. 115, 110501 ( 2015)]. In this paper, we theoretically obtain the minimal quantum entropy for three widely used linear dimension witnesses, which is considered "a difficult question." Then we experimentally test the classical and quantum entropy in a device-independent manner. The experimental results agree well with the theoretical analysis, demonstrating that entropy is needed in quantum systems that is lower than the entropy needed in classical systems with the given value of the dimension witness.
A Note on Mathai's Entropy Measure

Hans J. Haubold

2024-10-25

Abstract:In a paper [8] the authors classify entropy into three categories, as a thermodynamics quantity, as a measure of information production, and as a means of statistical inference. An entropy measure introduced by Mathai falls into the second and third categories. It is shown that this entropy measure is the same whether the variables involved are real or complex scalar, vector, or matrix variables. If the entropy measure is optimized under some moment-like conditions then one can obtain various types of densities which are applicable in different areas. Unlike Tsallis' entropy [9], it does not need an intermediary escort distribution to yield the desired results. Calculus of variation can be directly applied to obtain the desired results under Mathai's entropy. Tsallis' entropy, which is the basis of the area of non-extensive statistical mechanics, is a modified version of the ?-generalized entropy of Havrda-Charvat considered in [7]. Various types of distributions that can be obtained through optimization of Mathai's entropy, are illustrated in this paper.

Statistical Mechanics,Data Analysis, Statistics and Probability
What is Entropy?

John C. Baez

2024-09-14

Abstract:This short book is an elementary course on entropy, leading up to a calculation of the entropy of hydrogen gas at standard temperature and pressure. Topics covered include information, Shannon entropy and Gibbs entropy, the principle of maximum entropy, the Boltzmann distribution, temperature and coolness, the relation between entropy, expected energy and temperature, the equipartition theorem, the partition function, the relation between expected energy, free energy and entropy, the entropy of a classical harmonic oscillator, the entropy of a classical particle in a box, and the entropy of a classical ideal gas.

Statistical Mechanics,Mathematical Physics
Entropy, von Neumann and the von Neumann entropy

D. Petz

DOI: https://doi.org/10.48550/arXiv.math-ph/0102013

2001-02-10

Abstract:This paper is an introduction to the von Neumann entropy in a historic approach. Von Neumann's gedanken experiment is repeated, which led him to the formula of thermodynamic entropy of a statistical operator. In the analysis of his ideas we stress the role of superselection sectors and summarize von Neumann's knowledge about quantum mechanical entropy. The final part of the paper is devoted to important developments discovered long after von Neumann's work. Subadditivity and the interpretation of the von Neumann entropy as channel capacity are among those.

Mathematical Physics
Entropy Points And Applications

Xiangdong Ye,Guohua Zhang

DOI: https://doi.org/10.1090/S0002-9947-07-04357-7

IF: 1.3529

2007-01-01

Transactions of the American Mathematical Society

Abstract:First notions of entropy point and uniform entropy point are introduced using Bowen's definition of topological entropy. Some basic properties of the notions are discussed. As an application it is shown that for any topological dynamical system there is a countable closed subset whose Bowen entropy is equal to the entropy of the original system.Then notions of C-entropy point are introduced along the line of entropy tuple both in topological and measure-theoretical settings. It is shown that each C-entropy point is an entropy point, and the set of C-entropy points is the union of sets of C-entropy points for all invariant measures.
Measure-theoretic Entropy for Flows

Wenxiang Sun

DOI: https://doi.org/10.1007/bf02878695

1997-01-01

Science in China Series A Mathematics

Abstract:New definitions of measure-theoretic entropy for flows are given and the invariant property of the entropy under conjugacy is discussed.
Information and Helical Mechanism of Entropy Increase

Zou Dan Dan

DOI: https://doi.org/10.48550/arXiv.2102.11040

2021-02-19

Statistical Mechanics

Abstract:The principle of entropy increase is not only the basis of statistical mechanics, but also closely related to the irreversibility of time, the origin of life, chaos and turbulence. In this paper, we first discuss the dynamic system definition of entropy from the perspective of symbol and partition of information, and propose the entropy transfer characteristics based on the set partition. By introducing the hypothesis of limited accuracy of measurement into the continuous dynamical system, two necessary mechanisms for the formation of chaos are obtained: the transfer of entropy from small scale to macro scale (i.e. the increase of local entropy) and the dissipation of macro information. The relationship between the local entropy increase and Lyapunov exponent of dynamical system is established. And then the entropy increase and abnormal dissipation mechanism in physical system are analyzed and discussed.
Understanding Shannon's Entropy metric for Information

Sriram Vajapeyam

DOI: https://doi.org/10.48550/arXiv.1405.2061

2014-03-24

Information Theory

Abstract:Shannon's metric of "Entropy" of information is a foundational concept of information theory. This article is a primer for novices that presents an intuitive way of understanding, remembering, and/or reconstructing Shannon's Entropy metric for information.
Δ-Entropy: Definition, Properties and Applications in System Identification with Quantized Data

Badong Chen,Yu Zhu,Jinchun Hu,Jose C. Principe

DOI: https://doi.org/10.1016/j.ins.2010.11.037

IF: 8.1

2010-01-01

Information Sciences

Abstract:Recently, the minimum error entropy criterion, an information theoretic alternative to the traditional mean square error criterion, has been successfully used in the contexts of machine learning and signal processing. For system identification, however, the MEE criterion will be no longer suitable if the training data are discrete-valued, since minimizing error’s discrete entropy cannot constrain error’s dispersion. In this paper, to make the MEE criterion suitable for the discrete-valued data cases, we give a new entropy definition for the discrete random variables, i.e. the Δ-entropy, based on Riemann sums for finite size partitions. A probability weighted formula is established to calculate the average partition.This new entropy retains some important properties of the differential entropy and reduces to discrete entropy under certain conditions. Unlike discrete entropy, the Δ-entropy is sensitive to the dynamic range of the data, and can be used as a superior optimality criterion in system identification problems. Also, we present a plug-in estimate of Δ-entropy, analyze its asymptotic behavior and explore the links to the kernel based and m-spacing based estimates for differential entropy. Finally, the Δ-entropy criterion is applied in system identification with coarsely quantized input–output data to search for the optimum parameter set. Monte Carlo simulations demonstrate the performance improvement that may be achieved with the Δ-entropy criterion.
Interrelatedness of thermodynamics and information: transformation of heat as a measurable information process and quantity, an overview

B. Hejna,J. Šesták,Hejna, B.,Šesták, J.

DOI: https://doi.org/10.1007/s10973-023-12833-7

IF: 4.755

2024-03-05

Journal of Thermal Analysis and Calorimetry

Abstract:No one has yet investigated the information reach and influence of thermal analysis. Even so the heat and entropy are bound closely. The very problem of the term entropy is that it was defined specifically within the classical, equilibrium Thermodynamics like Clausius entropy, while its meaning is more general. It is usable in all areas of physics, in the communication theory and technology, and also, in social sciences, just as an universal quantitative model. The bridge to this wider world is in Boltzmann's statistical point of view used in the Shannon's information point of view. The difference is that the Shannon entropy is defined more generally, both for the uniform and for the not uniform probability distributions. But with renormalization and with the Boltzmann constant we can speak, using the Clausius entropy, about a certain thermodynamic system considered as a heat model of the area of our wider interests. So we will to deal with the heat and its entropy embodied to a wider consideration. In addition to the standard physical meaning, another thermodynamic realization in the form of a message is derived from Shannon's information theory, where the sample carries a measure of Shannon's information given by its structure or internal organization. This is also a contribution to improving the approach to thinking about the importance of observation and measurement, reflecting at the same time the costs and influence of the measurement itself and its organization on the measured object or, on its environment or, even on the environment of the measuring itself generally. Statistical sorting also carries with it the internal movement of individual elementary particles, the value of which indicates the resulting state of heat transfer and dissipation to and from the system providing a state of called equilibrium. It is statistically described by a weighted quantity called temperature, the changes of which via momentum alteration satisfy Newton's laws of inertia allowing enumeration of the related heat information processing. The terminology used is still non-institutional.

chemistry, physical, analytical,thermodynamics
Recent Progress in the Definition of Thermodynamic Entropy

Enzo Zanchini,Gian Paolo Beretta

DOI: https://doi.org/10.3390/e16031547

2014-03-24

Abstract:The principal methods for the definition of thermodynamic entropy are discussed with special reference to those developed by Carathéodory, the Keenan School, Lieb and Yngvason, and the present authors. An improvement of the latter method is then presented. Seven basic axioms are employed: three Postulates, which are considered as having a quite general validity, and four Assumptions, which identify the domains of validity of the definitions of energy (Assumption 1) and entropy (Assumptions 2, 3, 4). The domain of validity of the present definition of entropy is not restricted to stable equilibrium states. For collections of simple systems, it coincides with that of the proof of existence and uniqueness of an entropy function which characterizes the relation of adiabatic accessibility proposed by Lieb and Yngvason. However, our treatment does not require the formation of scaled copies so that it applies not only to collections of simple systems, but also to systems contained in electric or magnetic fields and to small and few-particle systems.

Mathematical Physics,Statistical Mechanics
The Measurement-Disturbance Relation and the Disturbance Trade-off Relation in Terms of Relative Entropy

Jun Zhang,Yang Zhang,Chang-shui Yu

DOI: https://doi.org/10.48550/arXiv.1704.04328

2017-04-14

Abstract:We employ quantum relative entropy to establish the relation between the measurement uncertainty and its disturbance on a state in the presence (and absence) of quantum memory. For two incompatible observables, we present the measurement-disturbance relation and the disturbance trade-off relation. We find that without quantum memory the disturbance induced by the measurement is never less than the measurement uncertainty and with quantum memory they depend on the conditional entropy of the measured state. We also generalize these relations to the case with multiple measurements. These relations are demonstrated by two examples.

Quantum Physics
Smallest Unit of Maximal Entropy as Novel Experimental Criterion for Parametric Characterization of Middle- and High-Entropy Materials

Alexander Khort,Alexander Dahlström,Sergey Roslyakov,Inger Odnevall

DOI: https://doi.org/10.1039/d4cp00776j

IF: 3.3

2024-03-28

Physical Chemistry Chemical Physics

Abstract:Materials with multiple principal elements (middle- and high-entropy materials), are used in emerging applications in various fields due to their unique properties, driven by configuration entropy. Improved understanding and experimental investigations of the impact of the entropy of mixing on the properties of these materials are of large practical interest. Here we show a simplified limited area calculation approach for assessing the entropy of mixing using a CoCuFeNi model nanoalloy. Based on our calculations we propose a new parametric entropy-based criterion, which defines critical scale parameter transition from the maximal entropy state to the entropy-depleted state of the system. The criterion could be used for generalized mechanistic assessment of the effect of the entropy of mixing on the characteristics of the materials with multiple principal elements and for the development and characterization of existing and new middle- and high-entropy materials with both simple single-, and more complex, multiply-sublattices structures.

chemistry, physical,physics, atomic, molecular & chemical
On dynamical measures of quantum information

James Fullwood,Arthur J. Parzygnat

2023-07-21

Abstract:In this work, we use the theory of quantum states over time to define an entropy $S(\rho,\mathcal{E})$ associated with quantum processes $(\rho,\mathcal{E})$, where $\rho$ is a state and $\mathcal{E}$ is a quantum channel responsible for the dynamical evolution of $\rho$. The entropy $S(\rho,\mathcal{E})$ is a generalization of the von Neumann entropy in the sense that $S(\rho,\mathrm{id})=S(\rho)$ (where $\mathrm{id}$ denotes the identity channel), and is a dynamical analogue of the quantum joint entropy for bipartite states. Such an entropy is then used to define dynamical formulations of the quantum conditional entropy and quantum mutual information, and we show such information measures satisfy many desirable properties, such as a quantum entropic Bayes' rule. We also use our entropy function to quantify the information loss/gain associated with the dynamical evolution of quantum systems, which enables us to formulate a precise notion of information conservation for quantum processes.

Quantum Physics,Information Theory,General Relativity and Quantum Cosmology

How and in What Sense Can the Entropy Be Measured?

Measurement-induced Nonlocality Based on the Relative Entropy

Dimensional measures of generalized entropy

Entropy of Quantum Measurements

An exploration for the macroscopic physical meaning of entropy

Entropy And Its Correlations With Other Related Quantities

Measure-theoretic metric mean dimension

Experimental Device-Independent Tests of Classical and Quantum Entropy

A Note on Mathai's Entropy Measure

What is Entropy?

Entropy, von Neumann and the von Neumann entropy

Entropy Points And Applications

Measure-theoretic Entropy for Flows

Information and Helical Mechanism of Entropy Increase

Understanding Shannon's Entropy metric for Information

Δ-Entropy: Definition, Properties and Applications in System Identification with Quantized Data

Interrelatedness of thermodynamics and information: transformation of heat as a measurable information process and quantity, an overview

Recent Progress in the Definition of Thermodynamic Entropy

The Measurement-Disturbance Relation and the Disturbance Trade-off Relation in Terms of Relative Entropy

Smallest Unit of Maximal Entropy as Novel Experimental Criterion for Parametric Characterization of Middle- and High-Entropy Materials

On dynamical measures of quantum information