Abstract:Many learning tasks are modeled as optimization problems with nonlinear constraints, such as principal component analysis and fitting a Gaussian mixture model. A popular way to solve such problems is resorting to Riemannian optimization algorithms, which yet heavily rely on both human involvement and expert knowledge about Riemannian manifolds. In this paper, we propose a Riemannian meta-optimization method to automatically learn a Riemannian optimizer. We parameterize the Riemannian optimizer by a novel recurrent network and utilize Riemannian operations to ensure that our method is faithful to the geometry of manifolds. The proposed method explores the distribution of the underlying data by minimizing the objective of updated parameters, and hence is capable of learning task-specific optimizations. We introduce a Riemannian implicit differentiation training scheme to achieve efficient training in terms of numerical stability and computational cost. Unlike conventional meta-optimization training schemes that need to differentiate through the whole optimization trajectory, our training scheme is only related to the final two optimization steps. In this way, our training scheme avoids the exploding gradient problem, and significantly reduces the computational load and memory footprint. We discuss experimental results across various constrained problems, including principal component analysis on Grassmann manifolds, face recognition, person re-identification, and texture image classification on Stiefel manifolds, clustering and similarity learning on symmetric positive definite manifolds, and few-shot learning on hyperbolic manifolds.

Solving Partial Least Squares Regression Via Manifold Optimization Approaches

Partial Least Squares Regression on Riemannian Manifolds and Its Application in Classifications

DAPLSR: Data Augmentation Partial Least Squares Regression Model via Manifold Optimization

Subsampling for Partial Least-Squares Regression Via an Influence Function

Author Response for "partial Least Median of Squares Regression"

Partial least trimmed squares regression

Multilinear and Nonlinear Generalizations of Partial Least Squares: an Overview of Recent Advances

Deep Learning Partial Least Squares

Robust methods for partial least squares regression

G-Optimal Feature Selection with Laplacian regularization

Jointly Sparse Global SIMPLS Regression

SIMPLS: An alternative approach to partial least squares regression

Mixed-norm Partial Least Squares

Optimal Weighting Distance-Based Similarity for Locally Weighted PLS Modeling

[HDDA] sparse subspace constrained partial least squares

Penalized Partial Least Squares Based on B-Splines Transformations

A Unified Parallel Algorithm for Regularized Group PLS Scalable to Big Data

Using the Krylov subspace formulation to improve regularisation and interpretation in partial least squares regression

Variance Reduced Stochastic Optimization for PCA and PLS.

Learning to Optimize on Riemannian Manifolds

Global Sparse Partial Least Squares