Abstract:Many learning tasks are modeled as optimization problems with nonlinear constraints, such as principal component analysis and fitting a Gaussian mixture model. A popular way to solve such problems is resorting to Riemannian optimization algorithms, which yet heavily rely on both human involvement and expert knowledge about Riemannian manifolds. In this paper, we propose a Riemannian meta-optimization method to automatically learn a Riemannian optimizer. We parameterize the Riemannian optimizer by a novel recurrent network and utilize Riemannian operations to ensure that our method is faithful to the geometry of manifolds. The proposed method explores the distribution of the underlying data by minimizing the objective of updated parameters, and hence is capable of learning task-specific optimizations. We introduce a Riemannian implicit differentiation training scheme to achieve efficient training in terms of numerical stability and computational cost. Unlike conventional meta-optimization training schemes that need to differentiate through the whole optimization trajectory, our training scheme is only related to the final two optimization steps. In this way, our training scheme avoids the exploding gradient problem, and significantly reduces the computational load and memory footprint. We discuss experimental results across various constrained problems, including principal component analysis on Grassmann manifolds, face recognition, person re-identification, and texture image classification on Stiefel manifolds, clustering and similarity learning on symmetric positive definite manifolds, and few-shot learning on hyperbolic manifolds.

Algorithm for Riemannian Manifold Learning

Learning an Image Manifold for Retrieval

Learning Exemplar-Represented Manifolds in Latent Space for Classification.

Riemannian manifold learning.

Deep Manifold Computing and Visualization

Isometric Immersion Learning with Riemannian Geometry

Deep Manifold Computing and Visualization Using Elastic Locally Isometric Smoothness

Deep Manifold Transformation for Nonlinear Dimensionality Reduction

Riemannian manifold learning for nonlinear dimensionality reduction

Applying Ricci Flow to High Dimensional Manifold Learning

Face Recognition Based on Riemannian Manifold Learning

Learning to Optimize on Riemannian Manifolds

Riemannian Metric Learning Based on Curvature Flow.

Adaptive Neighboring Selection Algorithm Based on Curvature Prediction in Manifold Learning

Locality-preserving Low-Rank Representation for Graph Construction from Nonlinear Manifolds

Active learning on manifolds

Invertible Manifold Learning for Dimension Reduction

Image-based 3D Model Retrieval Using Manifold Learning

A geometric viewpoint of manifold learning

A Regularized Approach for Geodesic-Based Semisupervised Multimanifold Learning

Curvature Flow Learning: Algorithm and Analysis.