Maximum entropy principle for Renyi's and Tsallis' entropies

A. G. Bashkirov,A.G.Bashkirov

DOI: https://doi.org/10.48550/arXiv.cond-mat/0310211

2003-10-09

Statistical Mechanics

Abstract:The equilibrium distributions of probabilities providing maximality of Renyi and Tsallis entropies are rederived. New S-forms of them are found which are normalised with corresponding entropies in contrast to the usual Z-forms normalised with partition functions.

What problem does this paper attempt to address?

The Statistical Discrepancy between the IGM and Dark Matter Fields: One-Point Statistics

Jesús Pando,Long-Long Feng,Li-Zhi Fang

DOI: https://doi.org/10.1086/422814

2004-05-26

Abstract:We investigate the relationship between the mass and velocity fields of the intergalactic medium (IGM) and dark matter. Although the evolution of the IGM is dynamically governed by the gravity of the underlying dark matter field, some statistical properties of the IGM inevitably decouple from those of the dark matter once the nonlinearity of the dynamical equations and the stochastic nature of the field is considered. With simulation samples produced by a hybrid cosmological hydrodynamic/N-body code, which is effective in capturing shocks and complicated structures with high precision, we find that the one-point distributions of the IGM field are systematically different from that of dark matter as follows: 1.) the one-point distribution of the IGM peculiar velocity field is exponential at least at redshifts less than 2, while the dark matter velocity field is close to a Gaussian field; 2.) although the one-point distributions of the IGM and dark matter are similar, the point-by-point correlation between the IGM and dark matter density fields significantly differs on all scales and redshifts analyzed; 3.) the one-point density distributions of the difference between IGM and dark matter fields are highly non-Gaussian and long tailed. These discrepancies violate the similarity between the IGM and dark matter and cannot be explained simply as Jeans smoothing of the IGM. However, these statistical discrepancies are consistent with the fluids described by stochastic-force driven nonlinear dynamics.

Astrophysics
On the foundations of the maximum entropy principle using Fenchel duality for Shannon and Tsallis entropies

Pierre Maréchal,Yasmín Navarrete,Sergio Davis

DOI: https://doi.org/10.1088/1402-4896/ad55b8

2024-06-08

Physica Scripta

Abstract:In this work, we address two main objectives. The first one is to provide a rigorous foundation to the maximum entropy principle in statistical physics, by making use of the Fenchel-Rockafellar duality. The second objective is to discuss the well-foundedness of the so-called escort distributions in the context of non-extensive entropy maximization. The duality treatment of maximum entropy confirms the non-rigorous results obtained via the usual variational calculus, however, the use of escort distributions yields undefined behavior when used consistently, and only leads to the desired results when used in an ad-hoc manner.

physics, multidisciplinary
Maximum Entropy Principle, Equal Probability a Priori and Gibbs Paradox

Hao Ge,Hong Qian

DOI: https://doi.org/10.48550/arxiv.1105.4118

2011-01-01

Abstract:We prove that information-theoretic maximum entropy (MaxEnt) approach to canonical ensemble is mathematically equivalent to the classic approach of Boltzmann, Gibbs and Darwin-Fowler. The two approaches, however, "interpret" a same mathematical theorem differently; most notably observing mean-energy in the former and energy conservation in the latter. However, applying the same MaxEnt method to grand canonical ensemble fails; while carefully following the classic approach based on Boltzmann's microcanonical {\em equal probability a priori} produces the correct statistics: One does not need to invoke quantum mechanics; and there is no Gibbs paradox. MaxEnt and related minimum relative entropy principle are based on the mathematical theorem concerning large deviations of rare fluctuations. As a scientific method, it requires classic mechanics, or some other assumptions, to provide meaningful {\em prior distributions} for the expected-value based statistical inference. A naive assumption of uniform prior is not valid in statistical mechanics.
Entropies for complex systems: generalized-generalized entropies

Stefan Thurner,Rudolf Hanel

DOI: https://doi.org/10.1063/1.2828761

2007-09-12

Abstract:Many complex systems are characterized by non-Boltzmann distribution functions of their statistical variables. If one wants to -- justified or not -- hold on to the maximum entropy principle for complex statistical systems (non-Boltzmann) we demonstrate how the corresponding entropy has to look like, given the form of the corresponding distribution functions. By two natural assumptions that (i) the maximum entropy principle should hold and that (ii) entropy should describe the correct thermodynamics of a system (which produces non-Boltzmann distributions) the existence of a class of fully consistent entropies can be deduced. Classical Boltzmann-Gibbs entropy is recovered as a special case for the observed distribution being the exponential, Tsallis entropy is the special case for q-exponential observations.

Statistical Mechanics
Nonsymmetric entropy I: basic concepts and results

Chengshi Liu

DOI: https://doi.org/10.48550/arXiv.cs/0611038

2006-11-09

Information Theory

Abstract:A new concept named nonsymmetric entropy which generalizes the concepts of Boltzman's entropy and shannon's entropy, was introduced. Maximal nonsymmetric entropy principle was proven. Some important distribution laws were derived naturally from maximal nonsymmetric entropy principle.
On the way towards a generalized entropy maximization procedure

G. Baris Bagci,Ugur Tirnakli

DOI: https://doi.org/10.1016/j.physleta.2009.07.014

IF: 2.707

2009-08-01

Physics Letters A

Abstract:We propose a generalized entropy maximization procedure, which takes into account the generalized averaging procedures and information gain definitions underlying the generalized entropies. This novel generalized procedure is then applied to Rényi and Tsallis entropies. The generalized entropy maximization procedure for Rényi entropies results in the exponential stationary distribution asymptotically for q∈(0,1] in contrast to the stationary distribution of the inverse power law obtained through the ordinary entropy maximization procedure. Another result of the generalized entropy maximization procedure is that one can naturally obtain all the possible stationary distributions associated with the Tsallis entropies by employing either ordinary or q-generalized Fourier transforms in the averaging procedure.

physics, multidisciplinary
Tsallis Entropy and the Transition to Scaling in Fragmentation

Oscar Sotolongo-Costa,Arezky Rodriguez,G. Rodgers

DOI: https://doi.org/10.3390/e2040172

IF: 2.738

2000-12-25

Entropy

Abstract:By using the maximum entropy principle with Tsallis entropy we obtain a fragment size distribution function which undergoes a transition to scaling. This distribution function reduces to those obtained by other authors using Shannon entropy. The treatment is easily generalisable to any process of fractioning with suitable constraints.

physics, multidisciplinary
Order Properties Concerning Tsallis Residual Entropy

Răzvan-Cornel Sfetcu,Vasile Preda

DOI: https://doi.org/10.3390/math12030417

IF: 2.4

2024-01-27

Mathematics

Abstract:With the help of Tsallis residual entropy, we introduce Tsallis quantile entropy order between two random variables. We give necessary and sufficient conditions, study closure and reversed closure properties under parallel and series operations and show that this order is preserved in the proportional hazard rate model, proportional reversed hazard rate model, proportional odds model and record values model.

mathematics
On the Completely Monotone Conjecture for Rényi Entropy

Hao Wu,Lei Yu,Laigang Guo

2023-12-04

Abstract:In this paper, we generalize the completely monotone conjecture from Shannon entropy to the Rényi entropy. We confirm this conjecture for the order of derivative up to $3$, when the order of Rényi entropy is in certain regimes. We also investigate concavity of Rényi entropy power and the completely monotone conjecture for Tsallis entropy. We observe that the completely monotone conjecture is true for Tsallis entropy of order $2$. Our proofs in this paper are based on the techniques of integration-by-parts and sum-of-squares.

Information Theory
A Modified Cosmic Brane Proposal for Holographic Renyi Entropy

Xi Dong,Jonah Kudler-Flam,Pratik Rath

DOI: https://doi.org/10.1007/JHEP06%282024%29120

2024-09-19

Abstract:We propose a new formula for computing holographic Renyi entropies in the presence of multiple extremal surfaces. Our proposal is based on computing the wave function in the basis of fixed-area states and assuming a diagonal approximation for the Renyi entropy. For Renyi index $n\geq1$, our proposal agrees with the existing cosmic brane proposal for holographic Renyi entropy. For $n<1$, however, our proposal predicts a new phase with leading order (in Newton's constant $G$) corrections to the cosmic brane proposal, even far from entanglement phase transitions and when bulk quantum corrections are unimportant. Recast in terms of optimization over fixed-area states, the difference between the two proposals can be understood to come from the order of optimization: for $n<1$, the cosmic brane proposal is a minimax prescription whereas our proposal is a maximin prescription. We demonstrate the presence of such leading order corrections using illustrative examples. In particular, our proposal reproduces existing results in the literature for the PSSY model and high-energy eigenstates, providing a universal explanation for previously found leading order corrections to the $n<1$ Renyi entropies.

High Energy Physics - Theory,Statistical Mechanics,General Relativity and Quantum Cosmology,Quantum Physics
Entropy Sequences and Maximal Entropy Sets

D Dou,XD Ye,GH Zhang

DOI: https://doi.org/10.1088/0951-7715/19/1/004

2005-01-01

Nonlinearity

Abstract:The notions of entropy sequences and entropy sets are introduced both in topological and measure-theoretical situations. It turns out that a subset is an entropy set if and only if each n-tuple (not on the diagonal) from the set is an entropy n-tuple and that each maximal entropy set is closed. The systems with only one maximal entropy set are characterized. Moreover, it is proved that if a topological system has positive entropy, then there is a maximal entropy set with uncountably many points, and the topological entropy is the supremum of the entropies over all maximal entropy sets. An example with a maximal entropy set containing only two points is given.
Fractal entropies from the Second Inverse Maximum Entropy Principle

Marcelo R. Ubriaco

2023-03-25

Abstract:With use of the Second Inverse Maximum Entropy Principle we find entropy functions for systems with fractal distribution functions with order parameter $q$. We compare these entropy functions with those given by the Bose-Einstein and Fermi-Dirac cases.

Statistical Mechanics
The dual formalisms of nonextensive thermodynamics for open systems with maximum entropy principle

Yahui Zheng,Haining Yu,Jiulin Du

DOI: https://doi.org/10.48550/arXiv.2002.10226

2020-02-03

Statistical Mechanics

Abstract:We study the nonextensive thermodynamics for open systems. On the basis of the maximum entropy principle, the dual power-law q-distribution functions are re-deduced by using the dual particle number definitions and assuming that the chemical potential is constant in the two sets of parallel formalisms, where the fundamental thermodynamic equations with dual interpretations of thermodynamic quantities are derived for the open systems. By introducing parallel structures of Legendre transformations, other thermodynamic equations with dual interpretations of quantities are also deduced in the open systems, and then several dual thermodynamic relations are inferred. One can easily find that there are correlations between the dual relations, from which an equivalent rule is found that the Tsallis factor is invariable in calculations of partial derivative with constant volume or constant entropy. Using this rule, more correlations can be found. And the statistical expressions of the Lagrange internal energy and pressure are easily obtained.
Conditional Entropy Formula with Respect to Monotonic Partitions

Weisheng Wu,Yiwei Zhang,Xiaomin Zhou

DOI: https://doi.org/10.1007/s10883-024-09682-4

IF: 1.27

2024-02-22

Journal of Dynamical and Control Systems

Abstract:We investigate conditional entropy with respect to monotonic (invariant, decreasing, or increasing) measurable partitions. In particular, we obtain Brin-Katok's and Katok's entropy formula for conditional entropy with respect to invariant, decreasing and a large class of increasing partitions. Our proofs develop new adaptions on applying conditional variants of Shannon-McMillan-Breiman theorems corresponding to the monotonic partitions, to link with the local and global conditional entropy forms. As an application, we obtain a number of reformulations of the unstable entropy for ergodic measures of general diffeomorphisms.

mathematics, applied,automation & control systems
Renyi extrapolation of Shannon entropy

Karol Zyczkowski

DOI: https://doi.org/10.48550/arXiv.quant-ph/0305062

2003-05-12

Quantum Physics

Abstract:Relations between Shannon entropy and Renyi entropies of integer order are discussed. For any N-point discrete probability distribution for which the Renyi entropies of order two and three are known, we provide an lower and an upper bound for the Shannon entropy. The average of both bounds provide an explicit extrapolation for this quantity. These results imply relations between the von Neumann entropy of a mixed quantum state, its linear entropy and traces.
Conditional entropies and their relation to entanglement criteria

Karl Gerd H. Vollbrecht,Michael M. Wolf

DOI: https://doi.org/10.1063/1.1498490

2002-02-11

Abstract:We discuss conditional Renyi and Tsallis entropies for bipartite quantum systems of finite dimension. We investigate the relation between the positivity of conditional entropies and entanglement properties. It is in particular shown that any state having a negative conditional entropy with respect to any value of the entropic parameter is distillable since it violates the reduction criterion. Moreover we show that the entanglement of Werner states in odd dimensions can neither be detected by entropic criteria nor by any other spectral criterion.

Quantum Physics
Maximum entropy and constraints in composite systems

John D Ramshaw,John D. Ramshaw

DOI: https://doi.org/10.1103/PhysRevE.105.024138

IF: 2.707

2022-02-25

Physical Review E

Abstract:The principle of maximum entropy (PME), as expounded by Jaynes, is based on the maximization of the Boltzmann-Gibbs-Shannon (BGS) entropy subject to linear constraints. The resulting probability distributions are of canonical (exponential) form. However, the rationale for linear constraints is nebulous, and probability distributions are not always canonical. Here we show that the correct noncanonical distribution for a system in equilibrium with a finite heat bath is implied by the unconstrained maximization of the total BGS entropy of the system and bath together. This procedure is shown to be equivalent to maximizing the BGS entropy of the system alone subject to a contrived nonlinear constraint which reduces to (a) the usual linear constraint for an infinite heat bath, and (b) a previously enigmatic logarithmic constraint which implies a power-law distribution for a large but finite heat bath. This procedure eliminates the uncertainty as to the proper constraints, and easily generalizes to arbitrary composite systems, for which it provides a simpler alternative to the Jaynes PME. https://doi.org/10.1103/PhysRevE.105.024138 ©2022 American Physical Society

physics, fluids & plasmas, mathematical
Impossible Mission: Entropy Maximization with Escort Averages

Thomas Oikonomou,G. Baris Bagci

DOI: https://doi.org/10.48550/arXiv.1704.04721

2018-03-22

Abstract:It has recently been a common practice to maximize the deformed entropies through the escort averaging scheme. However, whatever averaging procedure is employed, one should recover the ordinary Shannon maximization results in the appropriate limit of the deformation parameter e.g. $q\to 1$ for the Tsallis and Rényi entropies. Otherwise, the very meaning of a consistent generalization becomes at stake. Using only this equivalence, we show that any deformed entropy expression, maximized with the escort averaged constraints, yields that the Shannon entropy $S$ is equal to the logarithm of the ordinary canonical partition function i.e. $S = \ln(Z_S )$ instead of the correct thermodynamic relation $S = \beta U + \ln(Z_S )$. Therefore, we conclude that the use of the escort averaging procedure should be avoided for any deformed entropies, since it cannot even yield the well-known thermodynamic relations of the ordinary canonical formalism.

Statistical Mechanics
Maximum Geometric Quantum Entropy

Fabio Anza,James P. Crutchfield

DOI: https://doi.org/10.3390/e26030225

IF: 2.738

2024-03-02

Entropy

Abstract:Any given density matrix can be represented as an infinite number of ensembles of pure states. This leads to the natural question of how to uniquely select one out of the many, apparently equally-suitable, possibilities. Following Jaynes' information-theoretic perspective, this can be framed as an inference problem. We propose the Maximum Geometric Quantum Entropy Principle to exploit the notions of Quantum Information Dimension and Geometric Quantum Entropy. These allow us to quantify the entropy of fully arbitrary ensembles and select the one that maximizes it. After formulating the principle mathematically, we give the analytical solution to the maximization problem in a number of cases and discuss the physical mechanism behind the emergence of such maximum entropy ensembles.

physics, multidisciplinary

Maximum entropy principle for Renyi's and Tsallis' entropies

The Statistical Discrepancy between the IGM and Dark Matter Fields: One-Point Statistics

On the foundations of the maximum entropy principle using Fenchel duality for Shannon and Tsallis entropies

Maximum Entropy Principle, Equal Probability a Priori and Gibbs Paradox

Entropies for complex systems: generalized-generalized entropies

Nonsymmetric entropy I: basic concepts and results

On the way towards a generalized entropy maximization procedure

Tsallis Entropy and the Transition to Scaling in Fragmentation

Order Properties Concerning Tsallis Residual Entropy

On the Completely Monotone Conjecture for Rényi Entropy

A Modified Cosmic Brane Proposal for Holographic Renyi Entropy

Entropy Sequences and Maximal Entropy Sets

Fractal entropies from the Second Inverse Maximum Entropy Principle

The dual formalisms of nonextensive thermodynamics for open systems with maximum entropy principle

Conditional Entropy Formula with Respect to Monotonic Partitions

Renyi extrapolation of Shannon entropy

Conditional entropies and their relation to entanglement criteria

Maximum entropy and constraints in composite systems

Impossible Mission: Entropy Maximization with Escort Averages

Maximum Geometric Quantum Entropy