Axioms and uniqueness theorem for Tsallis entropy

Sumiyoshi Abe

DOI: https://doi.org/10.1016/S0375-9601%2800%2900337-6

2000-06-01

Abstract:The Shannon-Khinchin axioms for the ordinary information entropy are generalized in a natural way to the nonextensive systems based on the concept of nonextensive conditional entropy, and a complete proof of the uniqueness theorem for the Tsallis entropy is presented.

Statistical Mechanics

What problem does this paper attempt to address?

Generalization of Shannon-Khinchin Axioms to Nonextensive Systems and the Uniqueness Theorem

Hiroki Suyari

DOI: https://doi.org/10.48550/arXiv.math-ph/0205004

2002-05-29

Abstract:The Shannon-Khinchin axioms are generalized to nonextensive systems and the uniqueness theorem for the nonextensive entropy is proved rigorously. In the present axioms, Shannon additivity is used as additivity in contrast to pseudoadditivity in Abe's axioms. The results reveal that Tsallis entropy is the simplest among all nonextensive entropies which can be obtained from the generalized Shannon-Khinchin axioms.

Mathematical Physics
Generalized Shannon-Khinchin Axioms and Uniqueness Theorem for Pseudo-additive Entropies

Velimir M. Ilic,Miomir S. Stankovic

DOI: https://doi.org/10.1016/j.physa.2014.05.009

2013-11-02

Abstract:We consider the Shannon-Khinchin axiomatic systems for the characterization of generalized entropies such as Sharma-Mital and Frank-Daffertshofer entropy. We provide the generalization of Shannon-Khinchin axioms and give the corresponding uniqueness theorem. The previous attempts at such axiomatizations are also discussed.

Mathematical Physics
Tsallis entropy and generalized Shannon additivity

Sonja Jäckle,Karsten Keller

DOI: https://doi.org/10.48550/arXiv.1704.08011

2017-04-26

Abstract:The Tsallis entropy given for a positive parameter $\alpha$ can be considered as a modification of the classical Shannon entropy. For the latter, corresponding to $\alpha=1$, there exist many axiomatic characterizations. One of them based on the well-known Khinchin-Shannon axioms has been simplified several times and adapted to Tsallis entropy, where the axiom of (generalized) Shannon additivity is playing a central role. The main aim of this paper is to discuss this axiom in the context of Tsallis entropy. We show that it is sufficient for characterizing Tsallis entropy with the exceptions of cases $\alpha=1,2$ discussed separately.

Mathematical Physics
On the exact variance of Tsallis entropy in a random pure state

Lu Wei

DOI: https://doi.org/10.48550/arXiv.1807.02212

2018-07-06

Mathematical Physics

Abstract:Tsallis entropy is a useful one-parameter generalization of the standard von Neumann entropy in information theory. We study the variance of Tsallis entropy of bipartite quantum systems in a random pure state. The main result is an exact variance formula of Tsallis entropy that involves finite sums of some terminating hypergeometric functions. In the special cases of quadratic entropy and small subsystem dimensions, the main result is further simplified to explicit variance expressions. As a byproduct, we find an independent proof of the recently proved variance formula of von Neumann entropy based on the derived moment relation to the Tsallis entropy.
Tsallis eXtropy

Yige Xue,Yong Deng

DOI: https://doi.org/10.1080/03610926.2021.1921804

2021-05-10

Abstract:<span>The Tsallis entropy has high performance in handling uncertain information, which is the extension of information entropy. The extropy is a complementary dual of entropy, which is a research hot spot. This paper proposed the Tsallis eXtropy and maximum Tsallis eXtropy, which are the complementary of Tsallis entropy and maximum Tsallis entropy respectively. When the non-extensive parameter is equal to 1, then the Tsallis eXtropy and the maximum Tsallis eXtropy will degenerate into information extropy and maximum information extropy respectively. Some meaningful theorems and proofs of the Tsallis eXtropy and the maximum Tsallis eXtropy has been proposed. Numerical examples have been designed to prove the effectiveness of the proposed models in evaluating uncertainties. Comparison application has been applied to prove the superiority of the proposed models than other models. The experimental results have shown that the proposed models are very high effective in measuring unknown issues.</span>
Qualms concerning Tsallis's condition of pseudo-additivity as a definition of non-extensivity

B. H. Lavenda,J. Dunning-Davies

DOI: https://doi.org/10.48550/arXiv.cond-mat/0311477

2003-11-20

Abstract:The pseudo-additive relation that the Tsallis entropy satisfies has nothing whatsoever to do with the super- and sub- additivity properties of the entropy. The latter properties, like concavity and convexity, are couched in geometric inequalities and cannot be reduced to equalities. Rather, the pseudo-additivity relation is a functional equation that determines the functional forms of the random entropies. The Arimoto entropy satisfies a similar pseudo-additive relation and yet it is a first order homogeneous form. Hence no conclusions can be drawn on the extensive nature of the system from either the Tsallis or Arimoto entropy based on the pseudo-additive equation.

Statistical Mechanics
Entropy Maximization with Linear Constraints: The Uniqueness of the Shannon Entropy

Thomas Oikonomou,G. Baris Bagci

DOI: https://doi.org/10.48550/arXiv.1803.02556

2018-04-28

Abstract:Within a framework of utmost generality, we show that the entropy maximization procedure with linear constraints uniquely leads to the Shannon-Boltzmann-Gibbs entropy. Therefore, the use of this procedure with linear constraints should not be extended to the generalized entropies introduced recently. In passing, it is remarked how the forceful use of the entropy maximization for the Tsallis and Rényi entropies implies either the Shannon limit of these entropies or self-referential contradictions. Finally, we note that the utilization of the entropy maximization procedure with different averaging schemes is beyond the scope of this work.

Statistical Mechanics
A two-parameter entropy and its fundamental properties

Supriyo Dutta,Shigeru Furuichi,Partha Guha

DOI: https://doi.org/10.1142/S0129055X2130003X

2024-05-02

Abstract:This article proposes a new two-parameter generalized entropy, which can be reduced to the Tsallis and the Shannon entropy for specific values of its parameters. We develop a number of information-theoretic properties of this generalized entropy and divergence, for instance, the sub-additive property, strong sub-additive property, joint convexity, and information monotonicity. This article presents an exposit investigation on the information-theoretic and information-geometric characteristics of the new generalized entropy and compare them with the properties of the Tsallis and the Shannon entropy.

Mathematical Physics,Information Theory
Axiomatic Relation between Thermodynamic and Information-Theoretic Entropies

Mirjam Weilenmann,Lea Kraemer,Philippe Faist,Renato Renner

DOI: https://doi.org/10.1103/physrevlett.117.260601

IF: 8.6

2016-12-22

Physical Review Letters

Abstract:Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a measure of uncertainty. In this Letter, we connect these two notions of entropy, using an axiomatic framework for thermodynamics [E. H. Lieb and J. Yngvason Proc. R. Soc. 469, 20130408 (2013)]. In particular, we obtain a direct relation between the Clausius entropy and the Shannon entropy, or its generalization to quantum systems, the von Neumann entropy. More generally, we find that entropy measures relevant in nonequilibrium thermodynamics correspond to entropies used in one-shot information theory.

physics, multidisciplinary
A Characterization of Entropy as a Universal Monoidal Natural Transformation

Cheuk Ting Li

2024-04-14

Abstract:We show that the essential properties of entropy (monotonicity, additivity and subadditivity) are consequences of entropy being a monoidal natural transformation from the under category functor $-/\mathsf{LProb}_{\rho}$ (where $\mathsf{LProb}_{\rho}$ is category of $\rho$-th-power-summable probability distributions, $0<\rho<1$) to $\Delta_{\mathbb{R}}$. Moreover, the Shannon entropy can be characterized as the universal monoidal natural transformation from $-/\mathsf{LProb}_{\rho}$ to the category of integrally closed partially ordered abelian groups (a reflective subcategory of the lax-slice 2-category over $\mathsf{MonCat}_{\ell}$ in the 2-category of monoidal categories), providing a succinct characterization of Shannon entropy as a reflection arrow. We can likewise define entropy for every monoidal category with a monoidal structure on its under categories (e.g. the category of finite abelian groups, the category of finite inhabited sets, the category of finite dimensional vector spaces, and the augmented simplex category) via the reflection arrow. This implies that all these entropies over different categories are components of a single natural transformation (the unit of the idempotent monad), allowing us to connect these entropies in a natural manner. We also provide a universal characterization of the conditional Shannon entropy based on the chain rule which, unlike the characterization of information loss by Baez, Fritz and Leinster, does not require any continuity assumption.

Category Theory,Information Theory
On Uniqueness of Solutions to Conservation Laws Verifying a Single Entropy Condition

Sam G. Krupa,Alexis F. Vasseur

DOI: https://doi.org/10.1142/S0219891619500061

2018-02-09

Abstract:For hyperbolic systems of conservation laws, uniqueness of solutions is still largely open. We aim to expand the theory of uniqueness for systems of conservation laws. One difficulty is that many systems have only one entropy. This contrasts with scalar conservation laws, where many entropies exist. It took until 1994 to show that one entropy is enough to ensure uniqueness of solutions for the scalar conservation laws (see Panov [Mat. Zametki, 55(5):116--129, 159, 1994]). This single entropy result was proven again by De Lellis, Otto and Westdickenberg about 10 years later [Quart. Appl. Math., 62(4):687--700, 2004]. These two proofs both rely on the special connection between Hamilton--Jacobi equations and scalar conservation laws in one space dimension. However, this special connection does not extend to systems. In this paper, we prove the single entropy result for scalar conservation laws without using Hamilton--Jacobi. Our proof lays out new techniques that are promising for showing uniqueness of solutions in the systems case.

Analysis of PDEs
Three classes of nonextensive entropies characterized by Shannon additivity and pseudoadditivity

Hiroki Suyari

DOI: https://doi.org/10.48550/arXiv.math-ph/0205001

2002-05-29

Abstract:Nonextensive entropies are divided into three classes, each of which is characterized by Shannon additivity and pseudoadditivity. One of the three classes has properties of both additivities. The remaining classes have only one property of the two additivities, respectively. An example of nonextensive entropy is shown concretely for each class. In particular, one class is found to consist of only the Tsallis entropy. More precisely, the Tsallis entropy is proved to be uniquely determined by only these two additivities. The present classification using these two distinct additivities reveals unique characteristics of the Tsallis entropy.

Mathematical Physics
Equivalence of additive-combinatorial linear inequalities for Shannon entropy and differential entropy

Ashok Vardhan Makkuva,Yihong Wu

DOI: https://doi.org/10.48550/arXiv.1601.07498

2016-09-09

Abstract:This paper addresses the correspondence between linear inequalities of Shannon entropy and differential entropy for sums of independent group-valued random variables. We show that any balanced (with the sum of coefficients being zero) linear inequality of Shannon entropy holds if and only if its differential entropy counterpart also holds; moreover, any linear inequality for differential entropy must be balanced. In particular, our result shows that recently proved differential entropy inequalities by Kontoyiannis and Madiman \cite{KM14} can be deduced from their discrete counterparts due to Tao \cite{Tao10} in a unified manner. Generalizations to certain abelian groups are also obtained. Our proof of extending inequalities of Shannon entropy to differential entropy relies on a result of Rényi \cite{Renyi59} which relates the Shannon entropy of a finely discretized random variable to its differential entropy and also helps in establishing the entropy of the sum of quantized random variables is asymptotically equal to that of the quantized sum; the converse uses the asymptotics of the differential entropy of convolutions with weak additive noise.

Information Theory
Nonsymmetric entropy I: basic concepts and results

Chengshi Liu

DOI: https://doi.org/10.48550/arXiv.cs/0611038

2006-11-09

Information Theory

Abstract:A new concept named nonsymmetric entropy which generalizes the concepts of Boltzman's entropy and shannon's entropy, was introduced. Maximal nonsymmetric entropy principle was proven. Some important distribution laws were derived naturally from maximal nonsymmetric entropy principle.
Conditional Entropy Formula with Respect to Monotonic Partitions

Weisheng Wu,Yiwei Zhang,Xiaomin Zhou

DOI: https://doi.org/10.1007/s10883-024-09682-4

IF: 1.27

2024-02-22

Journal of Dynamical and Control Systems

Abstract:We investigate conditional entropy with respect to monotonic (invariant, decreasing, or increasing) measurable partitions. In particular, we obtain Brin-Katok's and Katok's entropy formula for conditional entropy with respect to invariant, decreasing and a large class of increasing partitions. Our proofs develop new adaptions on applying conditional variants of Shannon-McMillan-Breiman theorems corresponding to the monotonic partitions, to link with the local and global conditional entropy forms. As an application, we obtain a number of reformulations of the unstable entropy for ergodic measures of general diffeomorphisms.

mathematics, applied,automation & control systems
A note on Shannon entropy

Tomasz Sobieszek

DOI: https://doi.org/10.48550/arXiv.1203.3322

2012-03-15

Abstract:We present a somewhat different way of looking on Shannon entropy. This leads to an axiomatisation of Shannon entropy that is essentially equivalent to that of Fadeev. In particular we give a new proof of Fadeev theorem.

Information Theory
A Note on Mathai's Entropy Measure

Hans J. Haubold

2024-10-25

Abstract:In a paper [8] the authors classify entropy into three categories, as a thermodynamics quantity, as a measure of information production, and as a means of statistical inference. An entropy measure introduced by Mathai falls into the second and third categories. It is shown that this entropy measure is the same whether the variables involved are real or complex scalar, vector, or matrix variables. If the entropy measure is optimized under some moment-like conditions then one can obtain various types of densities which are applicable in different areas. Unlike Tsallis' entropy [9], it does not need an intermediary escort distribution to yield the desired results. Calculus of variation can be directly applied to obtain the desired results under Mathai's entropy. Tsallis' entropy, which is the basis of the area of non-extensive statistical mechanics, is a modified version of the ?-generalized entropy of Havrda-Charvat considered in [7]. Various types of distributions that can be obtained through optimization of Mathai's entropy, are illustrated in this paper.

Statistical Mechanics,Data Analysis, Statistics and Probability
Maximum entropy principle for Renyi's and Tsallis' entropies

A. G. Bashkirov,A.G.Bashkirov

DOI: https://doi.org/10.48550/arXiv.cond-mat/0310211

2003-10-09

Statistical Mechanics

Abstract:The equilibrium distributions of probabilities providing maximality of Renyi and Tsallis entropies are rederived. New S-forms of them are found which are normalised with corresponding entropies in contrast to the usual Z-forms normalised with partition functions.
On the Completely Monotone Conjecture for Rényi Entropy

Hao Wu,Lei Yu,Laigang Guo

2023-12-04

Abstract:In this paper, we generalize the completely monotone conjecture from Shannon entropy to the Rényi entropy. We confirm this conjecture for the order of derivative up to $3$, when the order of Rényi entropy is in certain regimes. We also investigate concavity of Rényi entropy power and the completely monotone conjecture for Tsallis entropy. We observe that the completely monotone conjecture is true for Tsallis entropy of order $2$. Our proofs in this paper are based on the techniques of integration-by-parts and sum-of-squares.

Information Theory
The subadditivity of quantum entropy in Gaussian quantum systems

Lijun Liu,Xiaozhen Ge,Shuming Cheng

DOI: https://doi.org/10.1016/j.rinp.2024.107548

IF: 4.565

2024-04-01

Results in Physics

Abstract:The entropy of a quantum state measures information or uncertainty contained in the quantum system and plays a crucial role in quantum information theory. Many entropic properties, such as the subadditivity, have been studied for the discrete system with a finite-dimensional state space, while much less is explored in the infinite-dimensional system. In this work, we study the entropic properties of the continuous system that models a collection of Gaussian modes or fields. In particular, we identify the parameter range for the Rényi, Tsallis, and Unified entropies of Gaussian states that they are additive, subadditive, and strong subadditive. We show that the Rényi entropies are subadditive, further giving rise to the subadditivity of the Tsallis and Unified entropies in certain parameter range. We also find that the Tsallis and Unified entropies are not strong subadditive, while the Rényi entropies with the order ranging from 1 to 2 are conjectured to admit the strong subadditivity. Our results are useful in studying quantum correlations and channel capacities in Gaussian systems.

physics, multidisciplinary,materials science

Axioms and uniqueness theorem for Tsallis entropy

Generalization of Shannon-Khinchin Axioms to Nonextensive Systems and the Uniqueness Theorem

Generalized Shannon-Khinchin Axioms and Uniqueness Theorem for Pseudo-additive Entropies

Tsallis entropy and generalized Shannon additivity

On the exact variance of Tsallis entropy in a random pure state

Tsallis eXtropy

Qualms concerning Tsallis's condition of pseudo-additivity as a definition of non-extensivity

Entropy Maximization with Linear Constraints: The Uniqueness of the Shannon Entropy

A two-parameter entropy and its fundamental properties

Axiomatic Relation between Thermodynamic and Information-Theoretic Entropies

A Characterization of Entropy as a Universal Monoidal Natural Transformation

On Uniqueness of Solutions to Conservation Laws Verifying a Single Entropy Condition

Three classes of nonextensive entropies characterized by Shannon additivity and pseudoadditivity

Equivalence of additive-combinatorial linear inequalities for Shannon entropy and differential entropy

Nonsymmetric entropy I: basic concepts and results

Conditional Entropy Formula with Respect to Monotonic Partitions

A note on Shannon entropy

A Note on Mathai's Entropy Measure

Maximum entropy principle for Renyi's and Tsallis' entropies

On the Completely Monotone Conjecture for Rényi Entropy

The subadditivity of quantum entropy in Gaussian quantum systems