Abstract:In this paper, we study asymptotic behaviors of continuous-time and discrete-time gradient flows of a ``lower-unbounded" convex function $f$ on a Hadamard manifold $M$, particularly, their convergence properties to the boundary $M^{\infty}$ at infinity of $M$. We establish a duality theorem that the infimum of the gradient-norm $\|\nabla f(x)\|$ of $f$ over $M$ is equal to the supremum of the negative of the recession function $f^{\infty}$ of $f$ over the boundary $M^{\infty}$, provided the infimum is positive. Further, the infimum and the supremum are obtained by the limits of the gradient flows of $f$, Our results feature convex-optimization ingredients of the moment-weight inequality for reductive group actions by Georgoulas, Robbin, and Salamon,and are applied to noncommutative optimization by Bürgisser et al. FOCS 2019. We show that the gradient descent of the Kempf-Ness function for an unstable orbit converges to a 1-parameter subgroup in the Hilbert-Mumford criterion, and the associated moment-map sequence converges to the mimimum-norm point of the moment polytope. We show further refinements for operator scaling -- the left-right action on a matrix tuple $A= (A_1,A_2,\ldots,A_N)$. We characterize the gradient-flow limit of operator scaling via a vector-space generalization of the classical Dulmage-Mendelsohn decomposition of a bipartite graph. Also, for a special case of $N = 2$, we reveal that this limit determines the Kronecker canonical form of matrix pencils $s A_1+A_2$.

Gradient and Hessian of functions with non-independent variables

Pathwise Derivatives for Multivariate Distributions

Gradient and Hessian of Joint Probability Function with Applications on Chance-Constrained Programs

Subdifferential characterization of probability functions under Gaussian distribution

Gradient descent for unbounded convex functions on Hadamard manifolds and its applications to scaling problems

Infinite-dimensional Hamilton-Jacobi equations for statistical inference on sparse graphs

Special Ricci-Hessian equations on Kähler manifolds

On the algebraic dependence of holonomic functions

Dimension-independent functional inequalities by tensorization and projection arguments

Derivatives of Multilinear Functions of Matrices

Holonomic and non-holonomic geometric models associated to the Gibbs-Helmholtz equation

Nonparametric estimation via partial derivatives

The Geometry of Differential Harnack Estimates

Relative gradient optimization of the Jacobian term in unsupervised deep learning

Hessian transport gradient flows

An Application of the Holonomic Gradient Method to the Neural Tangent Kernel

Monotonicity formulas and Hessian of the Green function

A geometry in the set of solutions to ill-posed linear problems with box constraints: Applications to probabilities on discrete sets

Hessian Chain Bracketing

Relations between nonsmooth vector variational inequalities and nonsmooth vector optimization problems on Hadamard manifold in terms of bifunction

Geometry in global coordinates in mechanics and optimal transport