Abstract:Matrix-valued optimization tasks, including those involving symmetric positive definite (SPD) matrices, arise in a wide range of applications in machine learning, data science and statistics. Classically, such problems are solved via constrained Euclidean optimization, where the domain is viewed as a Euclidean space and the structure of the matrices (e.g., positive definiteness) enters as constraints. More recently, geometric approaches that leverage parametrizations of the problem as unconstrained tasks on the corresponding matrix manifold have been proposed. While they exhibit algorithmic benefits in many settings, they cannot directly handle additional constraints, such as inequality or sparsity constraints. A remedy comes in the form of constrained Riemannian optimization methods, notably, Riemannian Frank-Wolfe and Projected Gradient Descent. However, both algorithms require potentially expensive subroutines that can introduce computational bottlenecks in practise. To mitigate these shortcomings, we introduce a class of structured regularizers, based on symmetric gauge functions, which allow for solving constrained optimization on the SPD manifold with faster unconstrained methods. We show that our structured regularizers can be chosen to preserve or induce desirable structure, in particular convexity and "difference of convex" structure. We demonstrate the effectiveness of our approach in numerical experiments.

Geometric statistics with subspace structure preservation for SPD matrices

Differential geometry with extreme eigenvalues in the positive semidefinite cone

Totally geodesic submanifolds in the manifold SPD of symmetric positive-definite real matrices

Dimensionality Reduction on SPD Manifolds: The Emergence of Geometry-Aware Methods

Scaling-rotation distance and interpolation of symmetric positive-definite matrices

Structured Regularization for Constrained Optimization on the SPD Manifold

Geometric Characteristics of the Wasserstein Metric on SPD(n) and Its Applications on Data Processing

A New Riemannian Structure in SPD(n)

Riemannian Local Mechanism for SPD Neural Networks

The Cholesky normal distribution for SPD matrices and inference for the mean

Kernel Methods on the Riemannian Manifold of Symmetric Positive Definite Matrices

Product Geometries on Cholesky Manifolds with Applications to SPD Manifolds

Geometry-aware Similarity Learning on SPD Manifolds for Visual Recognition

Log-Euclidean Kernels for Sparse Representation and Dictionary Learning

Schur's Positive-Definite Network: Deep Learning in the SPD cone with structure

Differentially Private Fréchet Mean on the Manifold of Symmetric Positive Definite (SPD) Matrices with log-Euclidean Metric

On Geometric Analysis of Affine Sparse Subspace Clustering

Subspace power method for symmetric tensor decomposition

Semantic And Neighborhood Preserving Dictionary Learning For Symmetric Positive-Definite Matrices

Kernel Sparse Subspace Clustering on Symmetric Positive Definite Manifolds