Abstract:Many learning tasks are modeled as optimization problems with nonlinear constraints, such as principal component analysis and fitting a Gaussian mixture model. A popular way to solve such problems is resorting to Riemannian optimization algorithms, which yet heavily rely on both human involvement and expert knowledge about Riemannian manifolds. In this paper, we propose a Riemannian meta-optimization method to automatically learn a Riemannian optimizer. We parameterize the Riemannian optimizer by a novel recurrent network and utilize Riemannian operations to ensure that our method is faithful to the geometry of manifolds. The proposed method explores the distribution of the underlying data by minimizing the objective of updated parameters, and hence is capable of learning task-specific optimizations. We introduce a Riemannian implicit differentiation training scheme to achieve efficient training in terms of numerical stability and computational cost. Unlike conventional meta-optimization training schemes that need to differentiate through the whole optimization trajectory, our training scheme is only related to the final two optimization steps. In this way, our training scheme avoids the exploding gradient problem, and significantly reduces the computational load and memory footprint. We discuss experimental results across various constrained problems, including principal component analysis on Grassmann manifolds, face recognition, person re-identification, and texture image classification on Stiefel manifolds, clustering and similarity learning on symmetric positive definite manifolds, and few-shot learning on hyperbolic manifolds.

Learning Riemannian Metrics

Towards Generalized and Efficient Metric Learning on Riemannian Manifold.

Riemannian Metric Learning via Optimal Transport

Riemannian manifold learning.

Non-linear Metric Learning Using Metric Tensor.

Metric Learning Via Penalized Optimization.

Fixed-Rank Supervised Metric Learning On Riemannian Manifold

Geometric Mean Metric Learning

Low-Rank Similarity Metric Learning In High Dimensions

Riemannian Metric Learning for Symmetric Positive Definite Matrices

Riemannian Metric Learning Based on Curvature Flow.

Robust Metric Learning on Grassmann Manifolds with Generalization Guarantees.

Algorithm for Riemannian Manifold Learning

Regression-Based Metric Learning

Metric Learning: A General Dimension Reduction Framework for Classification and Visualization.

A Riemannian Approach to Ground Metric Learning for Optimal Transport

Learning to Optimize on Riemannian Manifolds

Metric Learning from Relative Comparisons by Minimizing Squared Residual

Metric Learning from Imbalanced Data

A Novel Riemannian Metric Based on Riemannian Structure and Scaling Information for Fixed Low-Rank Matrix Completion

Intrinsic Metric Learning with Subspace Representation.