Nonsymmetric entropy I: basic concepts and results

Chengshi Liu

DOI: https://doi.org/10.48550/arXiv.cs/0611038

2006-11-09

Information Theory

Abstract:A new concept named nonsymmetric entropy which generalizes the concepts of Boltzman's entropy and shannon's entropy, was introduced. Maximal nonsymmetric entropy principle was proven. Some important distribution laws were derived naturally from maximal nonsymmetric entropy principle.

What problem does this paper attempt to address?

Maximal nonsymmetric entropy leads naturally to Zipf's law

Chengshi Liu

DOI: https://doi.org/10.48550/arXiv.cond-mat/0609344

2006-09-14

Disordered Systems and Neural Networks

Abstract:As the most fundamental empirical law, Zipf's law has been studied from many aspects. But its meaning is still an open problem. Some models have been constructed to explain Zipf's law. In the letter, a new concept named nonsymmetric entropy was introduced, maximizing nonsymmetric entropy leads naturally to Zipf's law.
Maximum entropy principle for Renyi's and Tsallis' entropies

A. G. Bashkirov,A.G.Bashkirov

DOI: https://doi.org/10.48550/arXiv.cond-mat/0310211

2003-10-09

Statistical Mechanics

Abstract:The equilibrium distributions of probabilities providing maximality of Renyi and Tsallis entropies are rederived. New S-forms of them are found which are normalised with corresponding entropies in contrast to the usual Z-forms normalised with partition functions.
Generalized Entropy from Mixing: Thermodynamics, Mutual Information and Symmetry Breaking

Fariel Shafee

DOI: https://doi.org/10.48550/arXiv.0906.2458

2009-06-13

Statistical Mechanics

Abstract:We review a new form of entropy suggested by us, with origin in mixing of states of systems due to interactions and deformations of phase cells. It is demonstrated that this nonextensive form also leads to asymmetric maximal entropy configurations unlike Shannon entropy. We discuss how, beginning with quantum entanglement of microsystems with one another and with the environment, one can obtain classical stochasticity for our form of entropy.
Non-Hermitian Generalization of Rényi Entropy

Daili Li,Chao Zheng

DOI: https://doi.org/10.3390/e24111563

IF: 2.738

2022-10-31

Entropy

Abstract:From their conception to present times, different concepts and definitions of entropy take key roles in a variety of areas from thermodynamics to information science, and they can be applied to both classical and quantum systems. Among them is the Rényi entropy. It is able to characterize various properties of classical information with a unified concise form. We focus on the quantum counterpart, which unifies the von Neumann entropy, max- and min-entropy, collision entropy, etc. It can only be directly applied to Hermitian systems because it usually requires that the density matrices is normalized. For a non-Hermitian system, the evolved density matrix may not be normalized; i.e., the trace can be larger or less than one as the time evolution. However, it is not well-defined for the Rényi entropy with a non-normalized probability distribution relevant to the density matrix of a non-Hermitian system, especially when the trace of the non-normalized density matrix is larger than one. In this work, we investigate how to describe the Rényi entropy for non-Hermitian systems more appropriately. We obtain a concisely and generalized form of α-Rényi entropy, which we extend the unified order-α from finite positive real numbers to zero and infinity. Our generalized α-Rényi entropy can be directly calculated using both of the normalized and non-normalized density matrices so that it is able to describe non-Hermitian entropy dynamics. We illustrate the necessity of our generalization by showing the differences between ours and the conventional Rényi entropy for non-Hermitian detuning two-level systems.

physics, multidisciplinary
A two-parameter entropy and its fundamental properties

Supriyo Dutta,Shigeru Furuichi,Partha Guha

DOI: https://doi.org/10.1142/S0129055X2130003X

2024-05-02

Abstract:This article proposes a new two-parameter generalized entropy, which can be reduced to the Tsallis and the Shannon entropy for specific values of its parameters. We develop a number of information-theoretic properties of this generalized entropy and divergence, for instance, the sub-additive property, strong sub-additive property, joint convexity, and information monotonicity. This article presents an exposit investigation on the information-theoretic and information-geometric characteristics of the new generalized entropy and compare them with the properties of the Tsallis and the Shannon entropy.

Mathematical Physics,Information Theory
A Characterization of Entropy as a Universal Monoidal Natural Transformation

Cheuk Ting Li

2024-04-14

Abstract:We show that the essential properties of entropy (monotonicity, additivity and subadditivity) are consequences of entropy being a monoidal natural transformation from the under category functor $-/\mathsf{LProb}_{\rho}$ (where $\mathsf{LProb}_{\rho}$ is category of $\rho$-th-power-summable probability distributions, $0<\rho<1$) to $\Delta_{\mathbb{R}}$. Moreover, the Shannon entropy can be characterized as the universal monoidal natural transformation from $-/\mathsf{LProb}_{\rho}$ to the category of integrally closed partially ordered abelian groups (a reflective subcategory of the lax-slice 2-category over $\mathsf{MonCat}_{\ell}$ in the 2-category of monoidal categories), providing a succinct characterization of Shannon entropy as a reflection arrow. We can likewise define entropy for every monoidal category with a monoidal structure on its under categories (e.g. the category of finite abelian groups, the category of finite inhabited sets, the category of finite dimensional vector spaces, and the augmented simplex category) via the reflection arrow. This implies that all these entropies over different categories are components of a single natural transformation (the unit of the idempotent monad), allowing us to connect these entropies in a natural manner. We also provide a universal characterization of the conditional Shannon entropy based on the chain rule which, unlike the characterization of information loss by Baez, Fritz and Leinster, does not require any continuity assumption.

Category Theory,Information Theory
Entropy Sequences and Maximal Entropy Sets

D Dou,XD Ye,GH Zhang

DOI: https://doi.org/10.1088/0951-7715/19/1/004

2005-01-01

Nonlinearity

Abstract:The notions of entropy sequences and entropy sets are introduced both in topological and measure-theoretical situations. It turns out that a subset is an entropy set if and only if each n-tuple (not on the diagonal) from the set is an entropy n-tuple and that each maximal entropy set is closed. The systems with only one maximal entropy set are characterized. Moreover, it is proved that if a topological system has positive entropy, then there is a maximal entropy set with uncountably many points, and the topological entropy is the supremum of the entropies over all maximal entropy sets. An example with a maximal entropy set containing only two points is given.
Origins of the Combinatorial Basis of Entropy

Robert K. Niven

DOI: https://doi.org/10.48550/arXiv.0708.1861

2007-08-14

Classical Physics

Abstract:The combinatorial basis of entropy, given by Boltzmann, can be written $H = N^{-1} \ln \mathbb{W}$, where $H$ is the dimensionless entropy, $N$ is the number of entities and $\mathbb{W}$ is number of ways in which a given realization of a system can occur (its statistical weight). This can be broadened to give generalized combinatorial (or probabilistic) definitions of entropy and cross-entropy: $H=\kappa (\phi(\mathbb{W}) +C)$ and $D=-\kappa (\phi(\mathbb{P}) +C)$, where $\mathbb{P}$ is the probability of a given realization, $\phi$ is a convenient transformation function, $\kappa$ is a scaling parameter and $C$ an arbitrary constant. If $\mathbb{W}$ or $\mathbb{P}$ satisfy the multinomial weight or distribution, then using $\phi(\cdot)=\ln(\cdot)$ and $\kappa=N^{-1}$, $H$ and $D$ asymptotically converge to the Shannon and Kullback-Leibler functions. In general, however, $\mathbb{W}$ or $\mathbb{P}$ need not be multinomial, nor may they approach an asymptotic limit. In such cases, the entropy or cross-entropy function can be {\it defined} so that its extremization ("MaxEnt'' or "MinXEnt"), subject to the constraints, gives the ``most probable'' (``MaxProb'') realization of the system. This gives a probabilistic basis for MaxEnt and MinXEnt, independent of any information-theoretic justification. This work examines the origins of the governing distribution $\mathbb{P}$.... (truncated)
Axioms and uniqueness theorem for Tsallis entropy

Sumiyoshi Abe

DOI: https://doi.org/10.1016/S0375-9601%2800%2900337-6

2000-06-01

Abstract:The Shannon-Khinchin axioms for the ordinary information entropy are generalized in a natural way to the nonextensive systems based on the concept of nonextensive conditional entropy, and a complete proof of the uniqueness theorem for the Tsallis entropy is presented.

Statistical Mechanics
The Statistical Discrepancy between the IGM and Dark Matter Fields: One-Point Statistics

Jesús Pando,Long-Long Feng,Li-Zhi Fang

DOI: https://doi.org/10.1086/422814

2004-05-26

Abstract:We investigate the relationship between the mass and velocity fields of the intergalactic medium (IGM) and dark matter. Although the evolution of the IGM is dynamically governed by the gravity of the underlying dark matter field, some statistical properties of the IGM inevitably decouple from those of the dark matter once the nonlinearity of the dynamical equations and the stochastic nature of the field is considered. With simulation samples produced by a hybrid cosmological hydrodynamic/N-body code, which is effective in capturing shocks and complicated structures with high precision, we find that the one-point distributions of the IGM field are systematically different from that of dark matter as follows: 1.) the one-point distribution of the IGM peculiar velocity field is exponential at least at redshifts less than 2, while the dark matter velocity field is close to a Gaussian field; 2.) although the one-point distributions of the IGM and dark matter are similar, the point-by-point correlation between the IGM and dark matter density fields significantly differs on all scales and redshifts analyzed; 3.) the one-point density distributions of the difference between IGM and dark matter fields are highly non-Gaussian and long tailed. These discrepancies violate the similarity between the IGM and dark matter and cannot be explained simply as Jeans smoothing of the IGM. However, these statistical discrepancies are consistent with the fluids described by stochastic-force driven nonlinear dynamics.

Astrophysics
Conditional Entropy Formula with Respect to Monotonic Partitions

Weisheng Wu,Yiwei Zhang,Xiaomin Zhou

DOI: https://doi.org/10.1007/s10883-024-09682-4

IF: 1.27

2024-02-22

Journal of Dynamical and Control Systems

Abstract:We investigate conditional entropy with respect to monotonic (invariant, decreasing, or increasing) measurable partitions. In particular, we obtain Brin-Katok's and Katok's entropy formula for conditional entropy with respect to invariant, decreasing and a large class of increasing partitions. Our proofs develop new adaptions on applying conditional variants of Shannon-McMillan-Breiman theorems corresponding to the monotonic partitions, to link with the local and global conditional entropy forms. As an application, we obtain a number of reformulations of the unstable entropy for ergodic measures of general diffeomorphisms.

mathematics, applied,automation & control systems
A covariant regulator for entanglement entropy: proofs of the Bekenstein bound and QNEC

Jonah Kudler-Flam,Samuel Leutheusser,Adel A. Rahman,Gautam Satishchandran,Antony J. Speranza

2023-12-13

Abstract:While von Neumann entropies for subregions in quantum field theory universally contain ultraviolet divergences, differences between von Neumann entropies are finite and well-defined in many physically relevant scenarios. We demonstrate that such a notion of entropy differences can be rigorously defined in quantum field theory in a general curved spacetime by introducing a novel, covariant regulator for the entropy based on the modular crossed product. This regulator associates a type II von Neumann algebra to each spacetime subregion, resulting in well-defined renormalized entropies. This prescription reproduces formulas for entropy differences that coincide with heuristic formulas widely used in the literature, and we prove that it satisfies desirable properties such as unitary invariance and concavity. As an application, we provide proofs of the Bekenstein bound and the quantum null energy condition, formulated directly in terms of vacuum-subtracted von Neumann entropies.

High Energy Physics - Theory,General Relativity and Quantum Cosmology,Quantum Physics
Entropy Maximization with Linear Constraints: The Uniqueness of the Shannon Entropy

Thomas Oikonomou,G. Baris Bagci

DOI: https://doi.org/10.48550/arXiv.1803.02556

2018-04-28

Abstract:Within a framework of utmost generality, we show that the entropy maximization procedure with linear constraints uniquely leads to the Shannon-Boltzmann-Gibbs entropy. Therefore, the use of this procedure with linear constraints should not be extended to the generalized entropies introduced recently. In passing, it is remarked how the forceful use of the entropy maximization for the Tsallis and Rényi entropies implies either the Shannon limit of these entropies or self-referential contradictions. Finally, we note that the utilization of the entropy maximization procedure with different averaging schemes is beyond the scope of this work.

Statistical Mechanics
An algebraic generalization of the entropy and its application to statistics

Martin Schlather

2024-04-09

Abstract:We define a general notion of entropy in elementary, algebraic terms. Based on that, weak forms of a scalar product and a distance measure are derived. We give basic properties of these quantities, generalize the Cauchy-Schwarz inequality, and relate our approach to the theory of scoring rules. Many supporting examples illustrate our approach and give new perspectives on established notions, such as the likelihood, the Kullback-Leibler divergence, the uncorrelatedness of random variables, the scalar product itself, the Tichonov regularization and the mutual information.

Spectral Theory
Signed Rényi Entropy and Quantum Second Laws

Adam Brandenburger,Pierfrancesco La Mura

2024-10-21

Abstract:We modify the Rényi (1961) axioms for entropy to apply to negative (``signed") measures as arise, for example, in phase-space representations of quantum mechanics. We obtain two new measures of (lack of) information about a system -- which we propose as signed analogs to classical Shannon entropy and classical Rényi entropy, respectively. We show that signed Rényi entropy witnesses non-classicality of a system. Specifically, a measure has at least one negative component if and only if signed Rényi $\alpha$-entropy is negative for some $\alpha > 1$. The corresponding non-classicality test does not work with signed Shannon entropy. We next show that signed Rényi $2k$-entropy, when $k$ is a positive integer, is Schur-concave. (An example shows that signed Shannon entropy is not Schur-concave.) We then establish an abstract quantum H-theorem for signed measures. We prove that signed Rényi $2k$-entropy is non-decreasing under classical (``decohering") evolution of a signed measure, where the latter could be a Wigner function or other phase-space representation of a quantum system. (An example shows that signed Shannon entropy may be non-monotonic.) We also provide a characterization of the Second Law for signed Rényi $2$-entropy in terms of what we call eventual classicalization of evolution of a system. We conclude with an argument that signed Rényi $2$-entropy of the Wigner function is constant under Moyal bracket evolution.

Quantum Physics,Mathematical Physics
Entropy, von Neumann and the von Neumann entropy

D. Petz

DOI: https://doi.org/10.48550/arXiv.math-ph/0102013

2001-02-10

Abstract:This paper is an introduction to the von Neumann entropy in a historic approach. Von Neumann's gedanken experiment is repeated, which led him to the formula of thermodynamic entropy of a statistical operator. In the analysis of his ideas we stress the role of superselection sectors and summarize von Neumann's knowledge about quantum mechanical entropy. The final part of the paper is devoted to important developments discovered long after von Neumann's work. Subadditivity and the interpretation of the von Neumann entropy as channel capacity are among those.

Mathematical Physics
Generalization of Shannon-Khinchin Axioms to Nonextensive Systems and the Uniqueness Theorem

Hiroki Suyari

DOI: https://doi.org/10.48550/arXiv.math-ph/0205004

2002-05-29

Abstract:The Shannon-Khinchin axioms are generalized to nonextensive systems and the uniqueness theorem for the nonextensive entropy is proved rigorously. In the present axioms, Shannon additivity is used as additivity in contrast to pseudoadditivity in Abe's axioms. The results reveal that Tsallis entropy is the simplest among all nonextensive entropies which can be obtained from the generalized Shannon-Khinchin axioms.

Mathematical Physics
Transcendental properties of entropy-constrained sets: Part II

Vjosa Blakaj,Chokri Manai

2023-09-14

Abstract:In this work, we address the question of the impossibility of certain single-letter formulas by exploiting the semi-algebraic nature of various entropy-constrained sets. The focus lies on studying the properties of the level sets of relative entropy, mutual information, and Rényi entropies. We analyze the transcendental structure of the set of states in which one of the aforementioned entropy quantities is fixed. Our results rule out (semi)algebraic single-shot characterizations of these entropy measures with bounded ancilla for both the classical and quantum cases.

Mathematical Physics,Quantum Physics
Sufficient conditions for non-zero entropy of closed relations

IZTOK BANIČ,RENE GRIL ROGINA,JUDY KENNEDY,VAN NALL

DOI: https://doi.org/10.1017/etds.2024.11

2024-02-16

Ergodic Theory and Dynamical Systems

Abstract:We introduce the notions of returns and well-aligned sets for closed relations on compact metric spaces and then use them to obtain non-trivial sufficient conditions for such a relation to have non-zero entropy. In addition, we give a characterization of finite relations with non-zero entropy in terms of Li–Yorke and DC2 chaos.

mathematics, applied

Nonsymmetric entropy I: basic concepts and results

Maximal nonsymmetric entropy leads naturally to Zipf's law

Maximum entropy principle for Renyi's and Tsallis' entropies

Generalized Entropy from Mixing: Thermodynamics, Mutual Information and Symmetry Breaking

Non-Hermitian Generalization of Rényi Entropy

A two-parameter entropy and its fundamental properties

A Characterization of Entropy as a Universal Monoidal Natural Transformation

Entropy Sequences and Maximal Entropy Sets

Origins of the Combinatorial Basis of Entropy

Axioms and uniqueness theorem for Tsallis entropy

The Statistical Discrepancy between the IGM and Dark Matter Fields: One-Point Statistics

Conditional Entropy Formula with Respect to Monotonic Partitions

A covariant regulator for entanglement entropy: proofs of the Bekenstein bound and QNEC

Entropy Maximization with Linear Constraints: The Uniqueness of the Shannon Entropy

An algebraic generalization of the entropy and its application to statistics

Signed Rényi Entropy and Quantum Second Laws

Entropy, von Neumann and the von Neumann entropy

Generalization of Shannon-Khinchin Axioms to Nonextensive Systems and the Uniqueness Theorem

Transcendental properties of entropy-constrained sets: Part II

Sufficient conditions for non-zero entropy of closed relations