Abstract:Riemannian convex optimization and minimax optimization have recently drawn considerable attention. Their appeal lies in their capacity to adeptly manage the non-convexity of the objective function as well as constraints inherent in the feasible set in the Euclidean sense. In this work, we delve into monotone Riemannian Variational Inequality Problems (RVIPs), which encompass both Riemannian convex optimization and minimax optimization as particular cases. In the context of Euclidean space, it is established that the last-iterates of both the extragradient (EG) and past extragradient (PEG) methods converge to the solution of monotone variational inequality problems at a rate of $O\left(\frac{1}{\sqrt{T}}\right)$ (Cai et al., 2022). However, analogous behavior on Riemannian manifolds remains an open question. To bridge this gap, we introduce the Riemannian extragradient (REG) and Riemannian past extragradient (RPEG) methods. We demonstrate that both exhibit $O\left(\frac{1}{\sqrt{T}}\right)$ last-iterate convergence. Additionally, we show that the average-iterate convergence of both REG and RPEG is $O\left(\frac{1}{T}\right)$, aligning with observations in the Euclidean case (Mokhtari et al., 2020). These results are enabled by judiciously addressing the holonomy effect so that additional complications in Riemannian cases can be reduced and the Euclidean proof inspired by the performance estimation problem (PEP) technique or the sum-of-squares (SOS) technique can be applied again.

Riemannian proximal gradient methods

Riemannian Proximal Gradient Methods (extended Version)

Proximal methods for structured nonsmooth optimization over Riemannian submanifolds

On the linear convergence rate of Riemannian proximal gradient method

An Inexact Riemannian Proximal Gradient Method

Riemannian Stochastic Proximal Gradient Methods for Nonsmooth Optimization over the Stiefel Manifold

Convergence and Trade-Offs in Riemannian Gradient Descent and Riemannian Proximal Point

A Riemannian Proximal Newton Method

The Proximal Gradient Method for Composite Optimization Problems on Riemannian Manifolds

Accelerated Bregman Proximal Gradient Methods for Relatively Smooth Convex Optimization

Nonsmooth Optimization over the Stiefel Manifold and Beyond: Proximal Gradient Method and Recent Variants

Decentralized projected Riemannian gradient method for smooth optimization on compact submanifolds

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Towards Riemannian Accelerated Gradient Methods

A Decentralized Proximal Gradient Tracking Algorithm for Composite Optimization on Riemannian Manifolds

An Extension of Fast Iterative Shrinkage‐thresholding Algorithm to Riemannian Optimization for Sparse Principal Component Analysis

An Extension of Fast Iterative Shrinkage-thresholding to Riemannian Optimization for Sparse Principal Component Analysis

A Riemannian Proximal Newton-CG Method

Extragradient Type Methods for Riemannian Variational Inequality Problems