A Hamiltonian approach to the gradient-flow equations in information geometry

Tatsuaki Wada,Antoni M. Scarfone

2024-07-17

Abstract:We have studied the gradient-flow equations in information geometry from a point-particle perspective. Based on the motion of a null (or light-like) particle in a curved space, we have rederived the Hamiltonians which describe the gradient-flows in information geometry.

Mathematical Physics

What problem does this paper attempt to address?

Huygens' equations and the gradient-flow equations in information geometry

Tatsuaki Wada,Antonio M. Scarfone,Hiroshi Matsuzoe

2023-08-09

Abstract:We revisit the relation between the gradient-flow equations and Hamilton's equations in information geometry. By regarding the gradient-flow equations as Huygens' equations in geometric optics, we have related the gradient flows to the geodesic flows induced by the geodesic Hamiltonian in an appropriate Riemannian geometry. The original evolution parameter $\textit{t}$ in the gradient-flow equations is related to the arc-length parameter in the associated Riemannian manifold by Jacobi-Maupertuis transformation. As a by-product, it is found the relation between the gradient-flow equation and replicator equations.

Information Theory,Statistical Mechanics
A Weyl geometric approach to the gradient-flow equations in information geometry

Tatsuaki Wada

2023-07-22

Abstract:The gradient-flow equations with respect to the potential functions in information geometry are reconsidered from the perspective of the Weyl integrable geometry. The pre-geodesic equations associated with the gradient-flow equations are regarded as the general pre-geodesic equations in the Weyl integrable geometry.

Mathematical Physics,Information Theory
On a Complex Hessian Flow

Weimin Sheng,Jiaxiang Wang

DOI: https://doi.org/10.2140/pjm.2019.300.159

2019-01-01

Pacific Journal of Mathematics

Abstract:We consider a gradient flow generated by a complex Hessian functional which is defined on compact Kahler manifolds. By setting up the a priori estimates of the admissible solutions, we prove the long-time existence of the solution to the flow and its convergence. Thus we show the functional admits a local minimal point in the space of admissible functions. As its application, we show the solvability of a class of complex Hessian equations.
Information Geometry of Dynamics on Graphs and Hypergraphs

Tetsuya J. Kobayashi,Dimitri Loutchko,Atsushi Kamimura,Shuhei A. Horiguchi,Yuki Sughiyama

2023-08-05

Abstract:We introduce a new information-geometric structure associated with the dynamics on discrete objects such as graphs and hypergraphs. The presented setup consists of two dually flat structures built on the vertex and edge spaces, respectively. The former is the conventional duality between density and potential, e.g., the probability density and its logarithmic form induced by a convex thermodynamic function. The latter is the duality between flux and force induced by a convex and symmetric dissipation function, which drives the dynamics of the density. These two are connected topologically by the homological algebraic relation induced by the underlying discrete objects. The generalized gradient flow in this doubly dual flat structure is an extension of the gradient flows on Riemannian manifolds, which include Markov jump processes and nonlinear chemical reaction dynamics as well as the natural gradient and mirror descent. The information-geometric projections on this doubly dual flat structure lead to information-geometric extensions of the Helmholtz-Hodge decomposition and the Otto structure in $L^{2}$ Wasserstein geometry. The structure can be extended to non-gradient nonequilibrium flows, from which we also obtain the induced dually flat structure on cycle spaces. This abstract but general framework can extend the applicability of information geometry to various problems of linear and nonlinear dynamics.

Information Theory,Differential Geometry,Statistics Theory,Chemical Physics,Data Analysis, Statistics and Probability
Mechanics of geodesics in Information geometry and Black Hole Thermodynamics

Sumanto Chanda,Tatsuaki Wada

DOI: https://doi.org/10.1142/S0219887824500981

2023-12-06

Abstract:In this article we shall discuss the theory of geodesics in information geometry, and an application in astrophysics. We will study how gradient flows in information geometry describe geodesics, explore the related mechanics by introducing a constraint, and apply our theory to Gaussian model and black hole thermodynamics. Thus, we demonstrate how deformation of gradient flows leads to more general Randers-Finsler metrics, describe Hamiltonian mechanics that derive from a constraint, and prove duality via canonical transformation. We also verified our theories for a deformation of the Gaussian model, and described dynamical evolution of flat metrics for Kerr and Reissner-Nordström black holes.

Information Theory
A Geometric flow towards hamiltonian stationary submanifolds

Jingyi Chen,Micah Warren

2024-09-24

Abstract:In this paper, we introduce a geometric flow for Lagrangian submanifolds in a Kähler manifold that stays in its initial Hamiltonian isotopy class and is a gradient flow for volume. The stationary solutions are the Hamiltonian stationary Lagrangian submanifolds. The flow is not strictly parabolic but it corresponds to a fourth order strictly parabolic scalar equation in the cotangent bundle of the submanifold via Weinstein's Lagrangian neighborhood theorem. For any compact initial Lagrangian immersion, we establish short-time existence, uniqueness, and higher order estimates when the second fundamental forms are uniformly bounded up to time $T$.

Differential Geometry,Analysis of PDEs
Hamilton--Jacobi equations for controlled gradient flows: cylindrical test functions

Giovanni Conforti,Richard C. Kraaij,Daniela Tonon

2024-01-15

Abstract:This work is the second part of a program initiated in <a class="link-https" data-arxiv-id="2111.13258" href="https://arxiv.org/abs/2111.13258">arXiv:2111.13258</a> aiming at the development of an intrinsic geometric well-posedness theory for Hamilton-Jacobi equations related to controlled gradient flow problems in metric spaces. Our main contribution is that of showing that the comparison principle proven therein implies a comparison principle for viscosity solutions relative to smoother Hamiltonians, acting on test functions that are mere cylindrical functions of the underling squared metric distance and whose rigorous definition is achieved from the Evolutional Variational Inequality formulation of gradient flows (EVI). In particular, the new Hamiltonians no longer require to work with test functions containing Tataru's distance. This substantial simplification paves the way for the development of a comprehensive existence theory.

Analysis of PDEs,Probability
Geometry of generalized fluid flows

Anton Izosimov,Boris Khesin

2023-10-13

Abstract:The Euler equation of an ideal (i.e. inviscid incompressible) fluid can be regarded, following V.Arnold, as the geodesic flow of the right-invariant $L^2$-metric on the group of volume-preserving diffeomorphisms of the flow domain. In this paper we describe the common origin and symmetry of generalized flows, multiphase fluids (homogenized vortex sheets), and conventional vortex sheets: they all correspond to geodesics on certain groupoids of multiphase diffeomorphisms. Furthermore, we prove that all these problems are Hamiltonian with respect to a Poisson structure on a dual Lie algebroid, generalizing the Hamiltonian property of the Euler equation on a Lie algebra dual.

Differential Geometry,Analysis of PDEs,Symplectic Geometry
Gradient Flows in Filtering and Fisher-Rao Geometry

Abhishek Halder,Tryphon T. Georgiou

DOI: https://doi.org/10.48550/arXiv.1710.00064

2017-10-30

Abstract:Uncertainty propagation and filtering can be interpreted as gradient flows with respect to suitable metrics in the infinite dimensional manifold of probability density functions. Such a viewpoint has been put forth in recent literature, and a systematic way to formulate and solve the same for linear Gaussian systems has appeared in our previous work where the gradient flows were realized via proximal operators with respect to Wasserstein metric arising in optimal mass transport. In this paper, we derive the evolution equations as proximal operators with respect to Fisher-Rao metric arising in information geometry. We develop the linear Gaussian case in detail and show that a template two step optimization procedure proposed earlier by the authors still applies. Our objective is to provide new geometric interpretations of known equations in filtering, and to clarify the implication of different choices of metric.

Optimization and Control,Systems and Control
Information geometry of diffeomorphism groups

Boris Khesin,Gerard Misiołek,Klas Modin

2024-11-06

Abstract:The study of diffeomorphism groups and their applications to problems in analysis and geometry has a long history. In geometric hydrodynamics, pioneered by V.~Arnold in the 1960s, one considers an ideal fluid flow as the geodesic motion on the infinite-dimensional group of volume-preserving diffeomorphisms of the fluid domain with respect to the metric defined by the kinetic energy. Similar considerations on the space of densities lead to a geometric description of optimal mass transport and the Kantorovich-Wasserstein metric. Likewise, information geometry associated with the Fisher-Rao metric and the Hellinger distance has an equally beautiful infinite-dimensional geometric description and can be regarded as a higher-order Sobolev analogue of optimal transportation. In this work we review various metrics on diffeomorphism groups relevant to this approach and introduce appropriate topology, smooth structures and dynamics on the corresponding infinite-dimensional manifolds. Our main goal is to demonstrate how, alongside topological hydrodynamics, Hamiltonian dynamics and optimal mass transport, information geometry with its elaborate toolbox has become yet another exciting field for applications of geometric analysis on diffeomorphism groups.

Differential Geometry,Information Theory,Mathematical Physics
Gradient Flows and Riemannian Structure in the Gromov-Wasserstein Geometry

Zhengxin Zhang,Ziv Goldfeld,Kristjan Greenewald,Youssef Mroueh,Bharath K. Sriperumbudur

2024-07-16

Abstract:The Wasserstein space of probability measures is known for its intricate Riemannian structure, which underpins the Wasserstein geometry and enables gradient flow algorithms. However, the Wasserstein geometry may not be suitable for certain tasks or data modalities. Motivated by scenarios where the global structure of the data needs to be preserved, this work initiates the study of gradient flows and Riemannian structure in the Gromov-Wasserstein (GW) geometry, which is particularly suited for such purposes. We focus on the inner product GW (IGW) distance between distributions on $\mathbb{R}^d$. Given a functional $\mathsf{F}:\mathcal{P}_2(\mathbb{R}^d)\to\mathbb{R}$ to optimize, we present an implicit IGW minimizing movement scheme that generates a sequence of distributions $\{\rho_i\}_{i=0}^n$, which are close in IGW and aligned in the 2-Wasserstein sense. Taking the time step to zero, we prove that the discrete solution converges to an IGW generalized minimizing movement (GMM) $(\rho_t)_t$ that follows the continuity equation with a velocity field $v_t\in L^2(\rho_t;\mathbb{R}^d)$, specified by a global transformation of the Wasserstein gradient of $\mathsf{F}$. The transformation is given by a mobility operator that modifies the Wasserstein gradient to encode not only local information, but also global structure. Our gradient flow analysis leads us to identify the Riemannian structure that gives rise to the intrinsic IGW geometry, using which we establish a Benamou-Brenier-like formula for IGW. We conclude with a formal derivation, akin to the Otto calculus, of the IGW gradient as the inverse mobility acting on the Wasserstein gradient. Numerical experiments validating our theory and demonstrating the global nature of IGW interpolations are provided.

Analysis of PDEs,Optimization and Control,Machine Learning
Gradient flows in asymmetric metric spaces and applications

Shin-ichi Ohta,Wei Zhao

2023-07-20

Abstract:This paper is devoted to the investigation of gradient flows in asymmetric metric spaces (for example, irreversible Finsler manifolds and Minkowski normed spaces) by means of discrete approximation. We study basic properties of curves and upper gradients in asymmetric metric spaces, and establish the existence of a curve of maximal slope, which is regarded as a gradient curve in the non-smooth setting. { Introducing} a natural convexity assumption on the potential function, { which is called the $(p,\lambda)$-convexity,} we also obtain some regularizing effects on the asymptotic behavior of curves of maximal slope. Applications include several existence results for gradient flows in Finsler manifolds, doubly nonlinear differential evolution equations on { infinite-dimensional Funk spaces}, and heat flow on compact Finsler manifolds.

Differential Geometry
Hessian transport gradient flows

Wuchen Li,Lexing Ying

DOI: https://doi.org/10.1007/s40687-019-0198-9

2019-10-28

Research in the Mathematical Sciences

Abstract:We derive new gradient flows of divergence functions in the probability space embedded with a class of Riemannian metrics. The Riemannian metric tensor is built from the transported Hessian operator of an entropy function. The new gradient flow is a generalized Fokker–Planck equation and is associated with a stochastic differential equation that depends on the reference measure. Several examples of Hessian transport gradient flows and the associated stochastic differential equations are presented, including the ones for the reverse Kullback–Leibler divergence, α\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\alpha $$\end{document}-divergence, Hellinger distance, Pearson divergence, and Jenson–Shannon divergence.

mathematics
Information Newton's flow: second-order optimization method in probability space

Yifei Wang,Wuchen Li

DOI: https://doi.org/10.48550/arXiv.2001.04341

2020-08-05

Abstract:We introduce a framework for Newton's flows in probability space with information metrics, named information Newton's flows. Here two information metrics are considered, including both the Fisher-Rao metric and the Wasserstein-2 metric. A known fact is that overdamped Langevin dynamics correspond to Wasserstein gradient flows of Kullback-Leibler (KL) divergence. Extending this fact to Wasserstein Newton's flows, we derive Newton's Langevin dynamics. We provide examples of Newton's Langevin dynamics in both one-dimensional space and Gaussian families. For the numerical implementation, we design sampling efficient variational methods in affine models and reproducing kernel Hilbert space (RKHS) to approximate Wasserstein Newton's directions. We also establish convergence results of the proposed information Newton's method with approximated directions. Several numerical examples from Bayesian sampling problems are shown to demonstrate the effectiveness of the proposed method.

Optimization and Control,Machine Learning
Hopfield Neural Network Flow: A Geometric Viewpoint

Abhishek Halder,Kenneth F. Caluya,Bertrand Travacca,Scott J. Moura

DOI: https://doi.org/10.48550/arXiv.1908.01270

2019-11-14

Abstract:We provide gradient flow interpretations for the continuous-time continuous-state Hopfield neural network (HNN). The ordinary and stochastic differential equations associated with the HNN were introduced in the literature as analog optimizers, and were reported to exhibit good performance in numerical experiments. In this work, we point out that the deterministic HNN can be transcribed into Amari's natural gradient descent, and thereby uncover the explicit relation between the underlying Riemannian metric and the activation functions. By exploiting an equivalence between the natural gradient descent and the mirror descent, we show how the choice of activation function governs the geometry of the HNN dynamics. For the stochastic HNN, we show that the so-called "diffusion machine", while not a gradient flow itself, induces a gradient flow when lifted in the space of probability measures. We characterize this infinite dimensional flow as the gradient descent of certain free energy with respect to a Wasserstein metric that depends on the geodesic distance on the ground manifold. Furthermore, we demonstrate how this gradient flow interpretation can be used for fast computation via recently developed proximal algorithms.

Optimization and Control,Machine Learning,Neural and Evolutionary Computing
On the Geometry of Saddle-Point Algorithms

AM BLOCH,RW BROCKETT,TS RATIU

DOI: https://doi.org/10.1109/cdc.1992.371168

1992-01-01

Abstract:There has been great deal of innovative work in recent years relating discrete algorithms to continuous flows. Of particular interest are flows which are gradient flows or Hamiltonian flows. Hamiltonian flows do not have asymptotically stable equilibria, but a restriction of the system to a certain set of variables may have such an equilibrium. In nonlinear optimization and game theory there is an interest in systems with saddle point equilibria. The authors show that certain flows with such equilibria can be both Hamiltonian and gradient and discuss the relationship of such flows with the gradient method for finding saddle points in nonlinear optimization problems. These results are compared with gradient flows associated with the Toda lattice.<>
Hyperbolic Gradient Flow: Evolution of Graphs in R^n+1

De-Xing Kong,Kefeng Liu

2010-01-01

Abstract:In this paper we introduce a new geometric flow — the hyperbolic gradientflow for graphs in the (n+1)-dimensional Euclidean space ℝ^n+1.This kind of flow is new and very natural to understand the geometry ofmanifolds. We particularly investigate the global existence of the evolution ofconvex hypersurfaces in ℝ^n+1 and the evolution of plane curves,and prove that, under the hyperbolic gradient flow, they converge to thehyperplane and the straight line, respectively, when t goes to the infinity.Our results show that the theory of shock waves of hyperbolic conservation lawscan be naturally applied to do surgery on manifolds. Some fundamental but openproblems are also given.
Hamilton's Gradient Estimates and A Monotonicity Formula for Heat Flows on Metric Measure Spaces

renjin jiang,huichun zhang

DOI: https://doi.org/10.1016/j.na.2015.08.011

IF: 1.743

2016-01-01

Nonlinear Analysis

Abstract:In this paper, we extend the Hamilton’s gradient estimates (Hamilton 1993) and a monotonicity formula of entropy (Ni 2004) for heat flows from smooth Riemannian manifolds to (non-smooth) metric measure spaces with appropriate Riemannian curvature-dimension condition.
Asymmetric Relaxations Through the Lens of Information Geometry

Alessandro Bravetti,Miguel Ángel García Ariza,Pablo Padilla

2024-02-22

Abstract:We introduce a novel perspective on Newton's Law of Cooling by framing it within the context of information geometry. By characterizing it as a specific representation of a gradient flow, we establish a connection with the well-known Horse-Carrot Theorem and reveal previously unexplored instances of asymmetric relaxations in the context of endoreversible processes. We present a completely general and practical criterion for predicting asymmetries based on the Amari-Chentsov tensor, whose applicability extends to both classical and quantum thermodynamics. Illustrative examples include faster cooling down of quantum ideal gases, and relaxations reminiscent of the Mpemba effect in classical ideal gases.

Mathematical Physics,Statistical Mechanics
The Gradient Flow of Higgs Pairs

Jiayu Li,Xi Zhang

DOI: https://doi.org/10.4171/jems/284

2011-01-01

Journal of the European Mathematical Society

Abstract:We consider the gradient flow of the Yang-Mills-Higgs functional of Higgs pairs on a Hermitian vector bundle (E, H-0) over a Kahler surface (M, omega), and study the asymptotic behavior of the heat flow for Higgs pairs at infinity. The main result is that the gradient flow with initial condition (A(0), phi(0)) converges, in an appropriate sense which takes into account bubbling phenomena, to a critical point (A(infinity,) phi(infinity)) of this functional. We also prove that the limiting Higgs pair (A(infinity), phi(infinity)) can be extended smoothly to a vector bundle E-infinity over (M, omega) and the isomorphism class of the limiting Higgs bundle (E-infinity, A(infinity), phi(infinity)) is given by the double dual of the graded Higgs sheaves associated to the Harder-Narasimhan-Seshadri filtration of the initial Higgs bundle (E, A(0), phi(0)).

A Hamiltonian approach to the gradient-flow equations in information geometry

Huygens' equations and the gradient-flow equations in information geometry

A Weyl geometric approach to the gradient-flow equations in information geometry

On a Complex Hessian Flow

Information Geometry of Dynamics on Graphs and Hypergraphs

Mechanics of geodesics in Information geometry and Black Hole Thermodynamics

A Geometric flow towards hamiltonian stationary submanifolds

Hamilton--Jacobi equations for controlled gradient flows: cylindrical test functions

Geometry of generalized fluid flows

Gradient Flows in Filtering and Fisher-Rao Geometry

Information geometry of diffeomorphism groups

Gradient Flows and Riemannian Structure in the Gromov-Wasserstein Geometry

Gradient flows in asymmetric metric spaces and applications

Hessian transport gradient flows

Information Newton's flow: second-order optimization method in probability space

Hopfield Neural Network Flow: A Geometric Viewpoint

On the Geometry of Saddle-Point Algorithms

Hyperbolic Gradient Flow: Evolution of Graphs in R^n+1

Hamilton's Gradient Estimates and A Monotonicity Formula for Heat Flows on Metric Measure Spaces

Asymmetric Relaxations Through the Lens of Information Geometry

The Gradient Flow of Higgs Pairs