Information Geometry and Statistical Manifold

Mashbat Suzuki

DOI: https://doi.org/10.48550/arXiv.1410.3369

2014-10-09

Abstract:We review basic notions in the field of information geometry such as Fisher metric on statistical manifold, $\alpha$-connection and corresponding curvature following Amari's work . We show application of information geometry to asymptotic statistical inference.

Statistics Theory,Differential Geometry,Probability

What problem does this paper attempt to address?

Homogeneous statistical manifolds

Jun-ichi Inoguchi,Yu Ohno

2024-08-03

Abstract:The methods of Information geometry have been glowing up to develop various subjects of theoretical physics, including quantum information systems. The present article has two purposes. The first one is to develop general theory of homogeneous statistical manifolds. In particular we construct explicit examples of homogeneous statistical manifolds of low dimension. The second purpose is to classify $3$-dimensional Lie groups admitting non-trivial conjugate symmetric left invariant statistical structure.

Differential Geometry,Mathematical Physics
A hall of statistical mirrors

Gabriel J. H. Khan,Jun Zhang

DOI: https://doi.org/10.4310/ajm.2022.v26.n6.a3

2021-09-28

Asian Journal of Mathematics

Abstract:The primary objects of study in information geometry are statistical manifolds, which are parametrized families of probability measures, induced with the Fisher-Rao metric and a pair of torsion-free conjugate connections. In recent work, the authors considered parametrized probability distributions as partially-flat statistical manifolds admitting torsion and showed that there is a complex to symplectic duality on the tangent bundles of such manifolds, based on the dualistic geometry of the underlying manifold. In this paper, we explore this correspondence further in the context of Hessian manifolds, in which case the conjugate connections are both curvature- and torsion-free, and the associated dual pair of spaces are K\"ahler manifolds. We focus on several key examples and their geometric features. In particular, we show that the moduli space of univariate normal distributions gives rise to a correspondence between the Siegel half-space and the Siegel-Jacobi space, which are spaces that appear in the context of automorphic forms.

Mathematics
Information Geometry for the Working Information Theorist

Kumar Vijay Mishra,M. Ashok Kumar,Ting-Kam Leonard Wong

2023-10-06

Abstract:Information geometry is a study of statistical manifolds, that is, spaces of probability distributions from a geometric perspective. Its classical information-theoretic applications relate to statistical concepts such as Fisher information, sufficient statistics, and efficient estimators. Today, information geometry has emerged as an interdisciplinary field that finds applications in diverse areas such as radar sensing, array signal processing, quantum physics, deep learning, and optimal transport. This article presents an overview of essential information geometry to initiate an information theorist, who may be unfamiliar with this exciting area of research. We explain the concepts of divergences on statistical manifolds, generalized notions of distances, orthogonality, and geodesics, thereby paving the way for concrete applications and novel theoretical investigations. We also highlight some recent information-geometric developments, which are of interest to the broader information theory community.

Information Theory,Machine Learning,Signal Processing,Differential Geometry
Information Geometry of Wasserstein Statistics on Shapes and Affine Deformations

Shun-ichi Amari,Takeru Matsuda

2024-06-25

Abstract:Information geometry and Wasserstein geometry are two main structures introduced in a manifold of probability distributions, and they capture its different characteristics. We study characteristics of Wasserstein geometry in the framework of Li and Zhao (2023) for the affine deformation statistical model, which is a multi-dimensional generalization of the location-scale model. We compare merits and demerits of estimators based on information geometry and Wasserstein geometry. The shape of a probability distribution and its affine deformation are separated in the Wasserstein geometry, showing its robustness against the waveform perturbation in exchange for the loss in Fisher efficiency. We show that the Wasserstein estimator is the moment estimator in the case of the elliptically symmetric affine deformation model. It coincides with the information-geometrical estimator (maximum-likelihood estimator) when the waveform is Gaussian. The role of the Wasserstein efficiency is elucidated in terms of robustness against waveform change.

Statistics Theory,Machine Learning
Information Geometry of the Thermodynamic Manifold

Limei Cao,Huafei Sun,Liping Wu

DOI: https://doi.org/10.1393/ncb/i2008-10628-1

2008-01-01

Abstract:We consider the a-geometric structure of the thermodynamic manifold S which characterizes a given physical system. We obtain the alpha-Gaussian curvature which is the generalization of the Riemannian Gaussian curvature. We set up a submanifold M which is a 1-flat submanifold of S. By introducing the Kullback-Leibler divergence, we get the optimal estimation point on M from a point on S by means of projection, and the optimal estimation point is unique. At last; we give an example to illustrate our results.
Nonparametric Information Geometry: From Divergence Function to Referential-Representational Biduality on Statistical Manifolds

Jun Zhang

DOI: https://doi.org/10.3390/e15125384

IF: 2.738

2013-12-04

Entropy

Abstract:Divergence functions are the non-symmetric “distance” on the manifold, Μθ, of parametric probability density functions over a measure space, (Χ,μ). Classical information geometry prescribes, on Μθ: (i) a Riemannian metric given by the Fisher information; (ii) a pair of dual connections (giving rise to the family of α-connections) that preserve the metric under parallel transport by their joint actions; and (iii) a family of divergence functions ( α-divergence) defined on Μθ x Μθ, which induce the metric and the dual connections. Here, we construct an extension of this differential geometric structure from Μθ (that of parametric probability density functions) to the manifold, Μ, of non-parametric functions on X, removing the positivity and normalization constraints. The generalized Fisher information and α-connections on M are induced by an α-parameterized family of divergence functions, reflecting the fundamental convex inequality associated with any smooth and strictly convex function. The infinite-dimensional manifold, M, has zero curvature for all these α-connections; hence, the generally non-zero curvature of M can be interpreted as arising from an embedding of Μθ into Μ. Furthermore, when a parametric model (after a monotonic scaling) forms an affine submanifold, its natural and expectation parameters form biorthogonal coordinates, and such a submanifold is dually flat for α = ± 1, generalizing the results of Amari’s α-embedding. The present analysis illuminates two different types of duality in information geometry, one concerning the referential status of a point (measurable function) expressed in the divergence function (“referential duality”) and the other concerning its representation under an arbitrary monotone scaling (“representational duality”).

Computer Science
Information Geometric Complexity of Entropic Motion on Curved Statistical Manifolds under Different Metrizations of Probability Spaces

Steven Gassner,Carlo Cafaro

DOI: https://doi.org/10.1142/S0219887819500828

2019-07-05

Abstract:We investigate the effect of different metrizations of probability spaces on the information geometric complexity of entropic motion on curved statistical manifolds. Specifically, we provide a comparative analysis based upon Riemannian geometric properties and entropic dynamical features of a Gaussian probability space where the two distinct dissimilarity measures between probability distributions are the Fisher-Rao information metric and the alpha-order entropy metric. In the former case, we observe an asymptotic linear temporal growth of the information geometric entropy (IGE) together with a fast convergence to the final state of the system. In the latter case, instead, we note an asymptotic logarithmic temporal growth of the IGE together with a slow convergence to the final state of the system. Finally, motivated by our findings, we provide some insights on a tradeoff between complexity and speed of convergence to the final state in our information geometric approach to problems of entropic inference.

Mathematical Physics
Cartan-Schouten metrics for information geometry and machine learning

Andre Diatta,Bakary Manga,Fatimata Sy

2024-08-28

Abstract:We study Cartan-Schouten metrics, explore invariant dual connections, and propose them as models for Information Geometry. Based on the underlying Riemannian barycenter and the biinvariant mean of Lie groups, we subsequently propose a new parametric mean for data science and machine learning which comes with several advantages compared to traditional tools such as the arithmetic mean, median, mode, expectation, least square method, maximum likelihood, linear regression. We call a metric on a Lie group, a Cartan-Schouten metric, if its Levi-Civita connection is biinvariant, so every 1-parameter subgroup through the unit is a geodesic. Except for not being left or right invariant in general, Cartan-Schouten metrics enjoy the same geometry as biinvariant metrics, since they share the same Levi-Civita connection. To bypass the non-invariance apparent drawback, we show that Cartan-Schouten metrics are completely determined by their value at the unit. We give an explicit formula for recovering them from their value at the unit, thus making them much less computationally demanding, compared to general metrics on manifolds. Furthermore, Lie groups with Cartan-Schouten metrics are complete Riemannian or pseudo-Riemannian manifolds. We give a complete characterization of Lie groups with Riemannian or Lorentzian Cartan-Schouten metrics. Cartan-Schouten metrics are in abundance on 2-nilpotent Lie groups. Namely, on every 2-nilpotent Lie group, there is a 1-1 correspondence between the set of left invariant metrics and that of Cartan-Schouten metrics.

Differential Geometry,Information Theory,Mathematical Physics,Probability,Statistics Theory
Biconnection Gravity as a Statistical Manifold

Damianos Iosifidis,Konstantinos Pallikaris

2023-08-23

Abstract:We formulate a bi-Connection Theory of Gravity whose Gravitational action consists of a recently defined mutual curvature scalar. Namely, we build a gravitational theory consisting of one metric and two affine connections, in a Metric-Affine Gravity setup. Consequently, coupling the two connections on an equal footing with matter, we show that the geometry of the resulting theory is, quite intriguingly, that of Statistical Manifold. This ultimately indicates a remarkable mathematical correspondence between Gravity and Information Geometry.

General Relativity and Quantum Cosmology,Information Theory,High Energy Physics - Theory
Dual Geometric Structures and Instability of Bivariate Weibull Statistical Manifold

Zhen-ning ZHANG,Li-mei CAO,Bing-qing KE

2014-01-01

Abstract:To investigate the stability of bivariate Weibull distribution from the viewpoint of information geometry,the set of all bivariate Weibull distributions was considered as a manifold which was called bivariate Weibull statistical manifold. By computing the Fisher information matrix,the α-connections,α-curvature tensors,α-scalar curvature,and dual geometric structures were obtained. Moreover,bivariate Weibull statistical manifold was dual flat and had constant sectional curvature for α = ± 1. Meanwhile,in virtue of the dual flat geometric structures,the instability of the geodesic spreads on this manifold was obtained via the divergence( or instability) of the Jacobi vector field.
The Fisher Geometry and Geodesics of the Multivariate Normals, without Differential Geometry

Brodie A. J. Lawson,Kevin Burrage,Kerrie Mengersen,Rodrigo Weber dos Santos

2023-06-02

Abstract:Choosing the Fisher information as the metric tensor for a Riemannian manifold provides a powerful yet fundamental way to understand statistical distribution families. Distances along this manifold become a compelling measure of statistical distance, and paths of shorter distance improve sampling techniques that leverage a sequence of distributions in their operation. Unfortunately, even for a distribution as generally tractable as the multivariate normal distribution, this information geometry proves unwieldy enough that closed-form solutions for shortest-distance paths or their lengths remain unavailable outside of limited special cases. In this review we present for general statisticians the most practical aspects of the Fisher geometry for this fundamental distribution family. Rather than a differential geometric treatment, we use an intuitive understanding of the covariance-induced curvature of this manifold to unify the special cases with known closed-form solution and review approximate solutions for the general case. We also use the multivariate normal information geometry to better understand the paths or distances commonly used in statistics (annealing, Wasserstein). Given the unavailability of a general solution, we also discuss the methods used for numerically obtaining geodesics in the space of multivariate normals, identifying remaining challenges and suggesting methodological improvements.

Statistics Theory,Computation
Dually affine Information Geometry modeled on a Banach space

Goffredo Chirco,Giovanni Pistone

2024-05-13

Abstract:In this chapter, we study Information Geometry from a particular non-parametric or functional point of view. The basic model is a probabilities subset usually specified by regularity conditions. For example, probability measures mutually absolutely continuous or probability densities with a given degree of smoothness. We construct a manifold structure by giving an atlas of charts as mappings from probabilities to a Banach space. The charts we use are quite peculiar in that we consider only instances where the transition mappings are affine. We chose a particular expression of the tangent and cotangent bundles in this affine setting.

Statistics Theory
Geometric Structures of Fisher Z Distribution

ZHONG Fengwei,SUN Huafei,ZHANG Zhenning

DOI: https://doi.org/10.3321/j.issn:1000-7857.2007.09.008

2007-01-01

Abstract:Fisher Z distribution is a widely used statistics distribution in practice. In this paper, the Fisher Z distribution is studied from the viewpoint of information geometry. A dual affine structure is constructed on it and a divergence is defined as a distance like measure. Furthermore, the Riemannian metric, α-connections and α-curvatures of the Fisher Z manifold are given. At last, an immersion from the Fisher Z distribution manifold into the affine space is obtained.
Histogram-Based Fisher Information Embedding for Manifolds Clustering and Visualization

Jian Zou,ChuanCai Liu,Yue Zhang

DOI: https://doi.org/10.1109/ccpr.2009.5344152

2009-01-01

Abstract:In this paper, a nonparametric histogram-based fisher information embedding method is presented for clustering and visualizing data sets with non-Euclidean geometric structures. It is on the assumption that each data set is derived from a probability distribution that can be characterized by a probability density function (PDF) lying on a statistical manifold. The dissimilarities between data sets can be quantified with the corresponding geodesic distances on statistical manifold from the views of information geometry. Our method is designed to convert data distribution information into simplex discretely, thereby a similarity measure can be adopted with simplex geometry, further clustering or visualizing original manifolds can be performed on a low-dimensional Euclidean subspace via manifold learning. Experiments on clustering Swiss Roll and s-curve submanifolds and reconstructing a sphere demonstrate the effectiveness of our method.
Geometrical Structures of Certain Class, of Statistical Manifolds

ZHANG Zhen-ning,SUN Hua-fei,ZHONG Feng-wei,张真宁,孙华飞,仲锋惟

2008-01-01

Abstract:The geometrical structures of the certain class of statistical manifolds are investigated. The geometrical metrics of such statistical manifolds are obtained, which includes the original geometrical metrics of S. Amari.
Information geometry of the power inverse Gaussian distribution

,Huafei Sun,Fengwei W. Zhong

2007-01-01

Applied Sciences

Abstract:The power inverse Gaussian distribution is a common distribution in reliability analysis and lifetime models. In this paper, we give the geometric structures of the power inverse Gaussian manifold from the viewpoint of information geometry. Firstly, we obtain the Fisher information matrix, Riemannian connections, Gaussian curvature of the power inverse Gaussian manifold. Then we consider the dual structure of this manifold and investigate the KullBack divergence. At last we give an immersion from the power inverse Gaussian manifold into an affine space. © Balkan Society of Geometers, Geometry Balkan Press 2007.
Information geometry of diffeomorphism groups

Boris Khesin,Gerard Misiołek,Klas Modin

2024-11-06

Abstract:The study of diffeomorphism groups and their applications to problems in analysis and geometry has a long history. In geometric hydrodynamics, pioneered by V.~Arnold in the 1960s, one considers an ideal fluid flow as the geodesic motion on the infinite-dimensional group of volume-preserving diffeomorphisms of the fluid domain with respect to the metric defined by the kinetic energy. Similar considerations on the space of densities lead to a geometric description of optimal mass transport and the Kantorovich-Wasserstein metric. Likewise, information geometry associated with the Fisher-Rao metric and the Hellinger distance has an equally beautiful infinite-dimensional geometric description and can be regarded as a higher-order Sobolev analogue of optimal transportation. In this work we review various metrics on diffeomorphism groups relevant to this approach and introduce appropriate topology, smooth structures and dynamics on the corresponding infinite-dimensional manifolds. Our main goal is to demonstrate how, alongside topological hydrodynamics, Hamiltonian dynamics and optimal mass transport, information geometry with its elaborate toolbox has become yet another exciting field for applications of geometric analysis on diffeomorphism groups.

Differential Geometry,Information Theory,Mathematical Physics
On warped product on information geometry: statistical manifolds and statistical models

Nicolás Martínez Alba,Olga Garatejo Escobar

2024-10-25

Abstract:We study a differential geometric construction, the warped product, on the background geometry for information theory. Divergences, dual structures and symmetric 3-tensor are studied under this construction, and we show that warped product of manifolds endow with such structure also is endowed with the same geometric notion. However, warped product does not preserve canonical divergences, which in particular shows that warped product lacks of meaning in the information theory setting.

Differential Geometry,Information Theory
Statistical Biharmonicity of Identity Maps

Ryu Ueno

2024-11-21

Abstract:The tension field of the identity map from a statistical manifold to a Riemannian statistical manifold, which shares the same Riemannian metric, is the Tchevychev vector field multiplied by negative one. We derive a new class of statistical manifolds that satisfy the semi-equiaffine condition based on the statistical biharmonicity of the identity map. Furthermore, we determine the statistical structures of this class, when the pair of the manifold and the Riemannian metric is a simply connected complete Riemannian manifold of constant curvature.

Differential Geometry
The Fisher-Rao geometry of CES distributions

Florent Bouchard,Arnaud Breloy,Antoine Collas,Alexandre Renaux,Guillaume Ginolhac

2023-10-02

Abstract:When dealing with a parametric statistical model, a Riemannian manifold can naturally appear by endowing the parameter space with the Fisher information metric. The geometry induced on the parameters by this metric is then referred to as the Fisher-Rao information geometry. Interestingly, this yields a point of view that allows for leveragingmany tools from differential geometry. After a brief introduction about these concepts, we will present some practical uses of these geometric tools in the framework of elliptical distributions. This second part of the exposition is divided into three main axes: Riemannian optimization for covariance matrix estimation, Intrinsic Cramér-Rao bounds, and classification using Riemannian distances.

Machine Learning,Applications

Information Geometry and Statistical Manifold

Homogeneous statistical manifolds

A hall of statistical mirrors

Information Geometry for the Working Information Theorist

Information Geometry of Wasserstein Statistics on Shapes and Affine Deformations

Information Geometry of the Thermodynamic Manifold

Nonparametric Information Geometry: From Divergence Function to Referential-Representational Biduality on Statistical Manifolds

Information Geometric Complexity of Entropic Motion on Curved Statistical Manifolds under Different Metrizations of Probability Spaces

Cartan-Schouten metrics for information geometry and machine learning

Biconnection Gravity as a Statistical Manifold

Dual Geometric Structures and Instability of Bivariate Weibull Statistical Manifold

The Fisher Geometry and Geodesics of the Multivariate Normals, without Differential Geometry

Dually affine Information Geometry modeled on a Banach space

Geometric Structures of Fisher Z Distribution

Histogram-Based Fisher Information Embedding for Manifolds Clustering and Visualization

Geometrical Structures of Certain Class, of Statistical Manifolds

Information geometry of the power inverse Gaussian distribution

Information geometry of diffeomorphism groups

On warped product on information geometry: statistical manifolds and statistical models

Statistical Biharmonicity of Identity Maps

The Fisher-Rao geometry of CES distributions