Information Geometry of the Gaussian Space

Giovanni Pistone

DOI: https://doi.org/10.48550/arXiv.1803.08135

2018-03-20

Probability

Abstract:We discuss the Pistone-Sempi exponential manifold on the finite-dimensional Gaussian space. We consider the role of the entropy, the continuity of translations, Poincar\'e-type inequalities, the generalized differentiability of probability densities of the Gaussian space.

What problem does this paper attempt to address?

On the Increments of A D-Dimensional Gaussian Process

ZY Lin,KS Hwang

DOI: https://doi.org/10.4134/jkms.2005.42.6.1215

2005-01-01

Journal of the Korean Mathematical Society

Abstract:In this paper we establish some results on the increments of a d-dimensional Gaussian process with the usual Euclidean norm. In particular we obtain the law of iterated logarithm and the Book-Shore type theorem for the increments of a d-dimensional Gaussian process, via estimating upper bounds and lower bounds of large deviation probabilities on the suprema of the d-dimensional Gaussian process.
Dually affine Information Geometry modeled on a Banach space

Goffredo Chirco,Giovanni Pistone

2024-05-13

Abstract:In this chapter, we study Information Geometry from a particular non-parametric or functional point of view. The basic model is a probabilities subset usually specified by regularity conditions. For example, probability measures mutually absolutely continuous or probability densities with a given degree of smoothness. We construct a manifold structure by giving an atlas of charts as mappings from probabilities to a Banach space. The charts we use are quite peculiar in that we consider only instances where the transition mappings are affine. We chose a particular expression of the tangent and cotangent bundles in this affine setting.

Statistics Theory
Information geometry of the power inverse Gaussian distribution

,Huafei Sun,Fengwei W. Zhong

2007-01-01

Applied Sciences

Abstract:The power inverse Gaussian distribution is a common distribution in reliability analysis and lifetime models. In this paper, we give the geometric structures of the power inverse Gaussian manifold from the viewpoint of information geometry. Firstly, we obtain the Fisher information matrix, Riemannian connections, Gaussian curvature of the power inverse Gaussian manifold. Then we consider the dual structure of this manifold and investigate the KullBack divergence. At last we give an immersion from the power inverse Gaussian manifold into an affine space. © Balkan Society of Geometers, Geometry Balkan Press 2007.
Examples of Application of Nonparametric Information Geometry to Statistical Physics

Giovanni Pistone

DOI: https://doi.org/10.48550/arXiv.1308.5312

2013-08-24

Statistics Theory

Abstract:We review a nonparametric version of Amari's Information Geometry in which the set of positive probability densities on a given sample space is endowed with an atlas of charts to form a differentiable manifold modeled on Orlicz Banach spaces. This nonparametric setting is used to discuss the setting of typical problems in Machine Learning and Statistical Physics, such as relaxed optimization, Kullback-Leibler divergence, Boltzmann entropy, Boltzmann equation
Translations in the exponential Orlicz space with Gaussian weight

Giovanni Pistone

DOI: https://doi.org/10.48550/arXiv.1708.09555

2017-08-31

Statistics Theory

Abstract:We study the continuity of space translations on non-parametric exponential families based on the exponential Orlicz space with Gaussian reference density.
Information geometry for types in the large-$n$ limit of random matrices

David Jekel

2025-01-01

Abstract:We study the interaction between entropy and Wasserstein distance in free probability theory. In particular, we give lower bounds for several versions of free entropy dimension along Wasserstein geodesics, as well as study their topological properties with respect to Wasserstein distance. We also study moment measures in the multivariate free setting, showing the existence and uniqueness of solutions for a regularized version of Santambrogio's variational problem. The role of probability distributions in these results is played by types, functionals which assign values not only to polynomial test functions, but to all real-valued logical formulas built from them using suprema and infima. We give an explicit counterexample showing that in the framework of non-commutative laws, the usual notion of probability distributions using only non-commutative polynomial test functions, one cannot obtain the desired large-$n$ limiting behavior for both Wasserstein distance and entropy simultaneously in random multi-matrix models.

Operator Algebras,Functional Analysis,Logic,Probability
Information Geometric Complexity of Entropic Motion on Curved Statistical Manifolds under Different Metrizations of Probability Spaces

Steven Gassner,Carlo Cafaro

DOI: https://doi.org/10.1142/S0219887819500828

2019-07-05

Abstract:We investigate the effect of different metrizations of probability spaces on the information geometric complexity of entropic motion on curved statistical manifolds. Specifically, we provide a comparative analysis based upon Riemannian geometric properties and entropic dynamical features of a Gaussian probability space where the two distinct dissimilarity measures between probability distributions are the Fisher-Rao information metric and the alpha-order entropy metric. In the former case, we observe an asymptotic linear temporal growth of the information geometric entropy (IGE) together with a fast convergence to the final state of the system. In the latter case, instead, we note an asymptotic logarithmic temporal growth of the IGE together with a slow convergence to the final state of the system. Finally, motivated by our findings, we provide some insights on a tradeoff between complexity and speed of convergence to the final state in our information geometric approach to problems of entropic inference.

Mathematical Physics
Information Geometry of the Probability Simplex: A Short Course

Giovanni Pistone

DOI: https://doi.org/10.33581/1561-4085-2020-23-2-221-242

2020-07-09

Nonlinear Phenomena in Complex Systems

Abstract:This set of notes is intended for a short course aiming to provide an (almost) self-contained and (almost) elementary introduction to the topic of Information Geometry (IG) of the probability simplex. Such a course can be considered an introduction to the original monograph by Amari and Nagaoka [1], and to the recent monographs by Amari [2] and by Ay, Jost, Lê, and Schwachhöfer [3]. The focus is on a non-parametric approach, that is, I consider the geometry of the full probability simplex and compare the IG formalism with what is classically done in Statistical Physics.
Information measures and geometry of the hyperbolic exponential families of Poincaré and hyperboloid distributions

Frank Nielsen,Kazuki Okamura

DOI: https://doi.org/10.1007/s41884-024-00158-9

2024-11-25

Abstract:We study various information-theoretic measures and the information geometry of the Poincaré distributions and the related hyperboloid distributions, and prove that their statistical mixture models are universal density estimators of smooth densities in hyperbolic spaces. The Poincaré and the hyperboloid distributions are two types of hyperbolic probability distributions defined using different models of hyperbolic geometry. Namely, the Poincaré distributions form a triparametric bivariate exponential family whose sample space is the hyperbolic Poincaré upper-half plane and natural parameter space is the open 3D convex cone of two-by-two positive-definite matrices. The family of hyperboloid distributions form another exponential family which has sample space the forward sheet of the two-sheeted unit hyperboloid modeling hyperbolic geometry. In the first part, we prove that all $f$-divergences between Poincaré distributions can be expressed using three canonical terms using Eaton's framework of maximal group invariance. We also show that the $f$-divergences between any two Poincaré distributions are asymmetric except when those distributions belong to a same leaf of a particular foliation of the parameter space. We report closed-form formula for the Fisher information matrix, the Shannon's differential entropy and the Kullback-Leibler divergence. and Bhattacharyya distances between such distributions using the framework of exponential families. In the second part, we state the corresponding results for the exponential family of hyperboloid distributions by highlighting a parameter correspondence between the Poincaré and the hyperboloid distributions. Finally, we describe a random generator to draw variates and present two Monte Carlo methods to stochastically estimate numerically $f$-divergences between hyperbolic distributions.

Information Theory
Images of Gaussian and other stochastic processes under closed, densely-defined, unbounded linear operators

Tadashi Matsumoto,T. J. Sullivan

DOI: https://doi.org/10.1142/s0219530524400025

IF: 1.9559

2024-02-21

Analysis and Applications

Abstract:Analysis and Applications, Ahead of Print. Gaussian Processes (GPs) are widely-used tools in spatial statistics and machine learning and the formulae for the mean function and covariance kernel of a GP [math] that is the image of another GP [math] under a linear transformation [math] acting on the sample paths of [math] are well known, almost to the point of being folklore. However, these formulae are often used without rigorous attention to technical details, particularly when [math] is an unbounded operator such as a differential operator, which is common in many modern applications. This note provides a self-contained proof of the claimed formulae for the case of a closed, densely-defined operator [math] acting on the sample paths of a square-integrable (not necessarily Gaussian) stochastic process. Our proof technique relies upon Hille's theorem for the Bochner integral of a Banach-valued random variable.

mathematics, applied
On the Global Gaussian Lipschitz Space

Liguang Liu,Peter Sjogren

DOI: https://doi.org/10.1017/s0013091516000390

2017-01-01

Proceedings of the Edinburgh Mathematical Society

Abstract:It is well known that the standard Lipschitz space in Euclidean space, with exponent a. (0, 1), can be characterized by means of the inequality |partial derivative P(t)f/partial derivative t| less than or similar to t(alpha-1), where P(t)f is the Poisson integral of the function f. There are two cases: one can either assume that the functions in the space are bounded, or one can not make such an assumption. In the setting of the Ornstein-Uhlenbeck semigroup in R-n, Gatto and Urbina defined a Lipschitz space by means of a similar inequality for the Ornstein-Uhlenbeck Poisson integral, considering bounded functions. In a preceding paper, the authors characterized that space by means of a Lipschitz-type continuity condition. The present paper defines a Lipschitz space in the same setting in a similar way, but now without the boundedness condition. Our main result says that this space can also be described by a continuity condition. The functions in this space turn out to have at most logarithmic growth at infinity.
Information Geometry for the Working Information Theorist

Kumar Vijay Mishra,M. Ashok Kumar,Ting-Kam Leonard Wong

2023-10-06

Abstract:Information geometry is a study of statistical manifolds, that is, spaces of probability distributions from a geometric perspective. Its classical information-theoretic applications relate to statistical concepts such as Fisher information, sufficient statistics, and efficient estimators. Today, information geometry has emerged as an interdisciplinary field that finds applications in diverse areas such as radar sensing, array signal processing, quantum physics, deep learning, and optimal transport. This article presents an overview of essential information geometry to initiate an information theorist, who may be unfamiliar with this exciting area of research. We explain the concepts of divergences on statistical manifolds, generalized notions of distances, orthogonality, and geodesics, thereby paving the way for concrete applications and novel theoretical investigations. We also highlight some recent information-geometric developments, which are of interest to the broader information theory community.

Information Theory,Machine Learning,Signal Processing,Differential Geometry
The exponential Orlicz space in quantum information geometry

Anna Jenčová

DOI: https://doi.org/10.1007/s41884-023-00097-x

2023-01-24

Information Geometry

Abstract:We review the construction of a quantum version of the exponential statistical manifold over the set of all faithful normal positive functionals on a von Neumann algebra. The construction is based on the relative entropy approach to state perturbation. We construct a quantum version of the exponential Orlicz space and discuss the properties of this space and its dual with respect to Kosaki Lpdocumentclass[12pt]{minimal}usepackage{amsmath}usepackage{wasysym}usepackage{amsfonts}usepackage{amssymb}usepackage{amsbsy}usepackage{mathrsfs}usepackage{upgreek}setlength{oddsidemargin}{-69pt}egin{document}$$L_p$$end{document}-spaces. We show that the constructed manifold admits a canonical divergence satisfying a Pythagorean relation. We also prove that the manifold structure is invariant under sufficient channels.
The Generalized Gaussian Minkowski Problem

Jiaqian Liu,Shengyu Tang

2024-07-20

Abstract:This article delves into the $L_p$ Minkowski problem within the framework of generalized Gaussian probability space. This type of probability space was initially introduced in information theory through the seminal works of Lutwak, Yang, and Zhang [49,50], as well as by Lutwak, Lv, Yang, and Zhang [45]. The primary focus of this article lies in examining the existence of this problem. While the variational method is employed to explore the necessary and sufficient conditions for the existence of the normalized Minkowski problem when $p \in \mathbb{R} \setminus \{0\}$, our main emphasis is on the existence of the generalized Gaussian Minkowski problem without the normalization requirement, particularly in the smooth category for $p \geq 1$.

Analysis of PDEs,Probability
Information Geometry of the Thermodynamic Manifold

Limei Cao,Huafei Sun,Liping Wu

DOI: https://doi.org/10.1393/ncb/i2008-10628-1

2008-01-01

Abstract:We consider the a-geometric structure of the thermodynamic manifold S which characterizes a given physical system. We obtain the alpha-Gaussian curvature which is the generalization of the Riemannian Gaussian curvature. We set up a submanifold M which is a 1-flat submanifold of S. By introducing the Kullback-Leibler divergence, we get the optimal estimation point on M from a point on S by means of projection, and the optimal estimation point is unique. At last; we give an example to illustrate our results.
Information Geometry of Power Spectral

SYED Feroz Shah,PENG Lin-yu,SUN Hua-fei,DUAN Xiao-min

DOI: https://doi.org/10.3969/j.issn.1001-0645.2013.03.023

2013-01-01

Abstract:The information geometric structure of power spectral is given based on information geometry theory.The set of power spectral is taken as a differential manifold.Moreover,a Riemannian metric is defined and the dual connections are introduced.Further,the geometric metrics of three stochastic process models are founded in this paper.
Information geometric duality of $ϕ$-deformed exponential families

Jan Korbel,Rudolf Hanel,Stefan Thurner

DOI: https://doi.org/10.3390/e21020112

2018-12-24

Abstract:In the world of generalized entropies---which, for example, play a role in physical systems with sub- and super-exponential phasespace growth per degree of freedom---there are two ways for implementing constraints in the maximum entropy principle: linear- and escort constraints. Both appear naturally in different contexts. Linear constraints appear e.g. in physical systems, when additional information about the system is available through higher moments. Escort distributions appear naturally in the context of multifractals and information geometry. It was shown recently that there exists a fundamental duality that relates both approaches on the basis of the corresponding deformed logarithms (deformed-log duality). Here we show that there exists another duality that arises in the context of information geometry, relating the Fisher information of $\phi$-deformed exponential families that correspond to linear constraints (as studied by J. Naudts), with those that are based on escort constraints (as studied by S.-I. Amari). We explicitly demonstrate this information geometric duality for the case of $(c,d)$-entropy that covers all situations that are compatible with the first three Shannon-Khinchin axioms, and that include Shannon, Tsallis, Anteneodo-Plastino entropy, and many more as special cases. Finally, we discuss the relation between the deformed-log duality and the information geometric duality, and mention that the escort distributions arising in the two dualities are generally different and only coincide for the case of the Tsallis deformation.

Mathematical Physics
A Characterization of the Gaussian Lipschitz Space and Sharp Estimates for the Ornstein-Uhlenbeck Poisson Kernel

Liguang Liu,Peter Sjogren

DOI: https://doi.org/10.4171/rmi/912

2016-01-01

Revista Matemática Iberoamericana

Abstract:The Gaussian Lipschitz space was defined by Gatto and Urbina, by means of the Ornstein-Uhlenbeck Poisson kernel. We give a characterization of this space in terms of a combination of ordinary Lipschitz continuity conditions. The main tools used in the proof are sharp estimates of the Ornstein-Uhlenbeck Poisson kernel and some of its derivatives.
Monte Carlo Information Geometry: The dually flat case

Frank Nielsen,Gaëtan Hadjeres

DOI: https://doi.org/10.48550/arXiv.1803.07225

2018-03-20

Abstract:Exponential families and mixture families are parametric probability models that can be geometrically studied as smooth statistical manifolds with respect to any statistical divergence like the Kullback-Leibler (KL) divergence or the Hellinger divergence. When equipping a statistical manifold with the KL divergence, the induced manifold structure is dually flat, and the KL divergence between distributions amounts to an equivalent Bregman divergence on their corresponding parameters. In practice, the corresponding Bregman generators of mixture/exponential families require to perform definite integral calculus that can either be too time-consuming (for exponentially large discrete support case) or even do not admit closed-form formula (for continuous support case). In these cases, the dually flat construction remains theoretical and cannot be used by information-geometric algorithms. To bypass this problem, we consider performing stochastic Monte Carlo (MC) estimation of those integral-based mixture/exponential family Bregman generators. We show that, under natural assumptions, these MC generators are almost surely Bregman generators. We define a series of dually flat information geometries, termed Monte Carlo Information Geometries, that increasingly-finely approximate the untractable geometry. The advantage of this MCIG is that it allows a practical use of the Bregman algorithmic toolbox on a wide range of probability distribution families. We demonstrate our approach with a clustering task on a mixture family manifold.

Machine Learning

Information Geometry of the Gaussian Space

On the Increments of A D-Dimensional Gaussian Process

Dually affine Information Geometry modeled on a Banach space

Information geometry of the power inverse Gaussian distribution

Examples of Application of Nonparametric Information Geometry to Statistical Physics

Translations in the exponential Orlicz space with Gaussian weight

Information geometry for types in the large-$n$ limit of random matrices

Information Geometric Complexity of Entropic Motion on Curved Statistical Manifolds under Different Metrizations of Probability Spaces

Information Geometry of the Probability Simplex: A Short Course

Information measures and geometry of the hyperbolic exponential families of Poincaré and hyperboloid distributions

Images of Gaussian and other stochastic processes under closed, densely-defined, unbounded linear operators

On the Global Gaussian Lipschitz Space

Information Geometry for the Working Information Theorist

The exponential Orlicz space in quantum information geometry

The Generalized Gaussian Minkowski Problem

Information Geometry of the Thermodynamic Manifold

Information Geometry of Power Spectral

Information geometric duality of $ϕ$-deformed exponential families

A Characterization of the Gaussian Lipschitz Space and Sharp Estimates for the Ornstein-Uhlenbeck Poisson Kernel

Monte Carlo Information Geometry: The dually flat case