Abstract:This paper considers the problem of minimizing the summation of a differentiable function and a nonsmooth function on a Riemannian manifold. In recent years, proximal gradient method and its invariants have been generalized to the Riemannian setting for solving such problems. Different approaches to generalize the proximal mapping to the Riemannian setting lead versions of Riemannian proximal gradient methods. However, their convergence analyses all rely on solving their Riemannian proximal mapping exactly, which is either too expensive or impracticable. In this paper, we study the convergence of an inexact Riemannian proximal gradient method. It is proven that if the proximal mapping is solved sufficiently accurately, then the global convergence and local convergence rate based on the Riemannian Kurdyka-Łojasiewicz property can be guaranteed. Moreover, practical conditions on the accuracy for solving the Riemannian proximal mapping are provided. As a byproduct, the proximal gradient method on the Stiefel manifold proposed in~[CMSZ2020] can be viewed as the inexact Riemannian proximal gradient method provided the proximal mapping is solved to certain accuracy. Finally, numerical experiments on sparse principal component analysis are conducted to test the proposed practical conditions.
What problem does this paper attempt to address?
### What problem does this paper attempt to solve?
This paper mainly studies the Inexact Riemannian Proximal Gradient Method (IRPG) for composite optimization problems with smooth and non - smooth terms on Riemannian manifolds. Specifically, it considers the following form of optimization problem:
\[
\min_{x \in M} F(x)=f(x)+g(x),
\]
where \(M\) is a finite - dimensional Riemannian manifold, \(f\) is a differentiable function, and \(g\) is a continuous but possibly non - smooth function. This problem is very important in many practical applications, such as compressed models, sparse principal component analysis, cosparse factor regression, etc.
### Main contributions
1. **Proposed a general framework**: The paper proposed a general framework of the Inexact Riemannian Proximal Gradient Method (IRPG) and carried out theoretical analysis in terms of global convergence and local convergence rate respectively.
2. **Guaranteed global convergence**: The paper proved that if the Riemannian proximal mapping is solved with sufficient accuracy, global convergence can be guaranteed. In particular, when \(g\) is retraction - convex, a practical condition is provided to meet the requirements of local convergence rate analysis.
3. **Relationship with existing methods**: The paper pointed out that the previously proposed ManPG method can be regarded as an IRPG method in a special case, that is, global convergence can be guaranteed without accurately solving the proximal mapping.
4. **Numerical experiment verification**: The effectiveness of the proposed practical conditions was verified through numerical experiments of sparse principal component analysis.
### Key technical challenges
- **Non - linear constraints**: Due to the non - linearity of the manifold, developing the Riemannian proximal gradient method becomes complicated.
- **Computational complexity**: Accurately solving the Riemannian proximal mapping is usually very expensive or infeasible, so it is of great significance to study the convergence of inexact methods.
### Theoretical analysis
- **Global convergence**: Based on different theoretical conditions, the paper studied global convergence and local convergence rate respectively. Local convergence analysis depends on the Riemannian Kurdyka - Łojasiewicz (KL) property.
- **Local convergence rate**: Under more stringent accuracy conditions, the paper gave a local convergence rate analysis based on the Riemannian KL property.
### Practical conditions
To ensure the global convergence and local convergence rate of the algorithm, the paper provided specific practical conditions. These conditions are based on the definition of error bounds and are applicable to manifolds with linear ambient spaces.
In general, this paper provides a new and effective tool for solving composite optimization problems with manifold constraints by introducing the Inexact Riemannian Proximal Gradient Method, and theoretically proves its convergence and convergence speed.