Modified Armijo line-search in Riemannian optimization with reduced computational cost

Yuya Yamakawa,Hiroyuki Sato,Kensuke Aihara

2023-04-05

Abstract:In this paper, we propose a new line-search method that improves the ordinary Armijo line-search in Riemannian optimization. For optimization problems on Riemannian manifolds, many types of globally convergent algorithms have been proposed, and they are often equipped with the Armijo line-search in Riemannian optimization for global convergence. Such existing methods need the computation of a retraction regarding the search direction for each iteration. On the other hand, the proposed line-search decreases the computational cost by incorporating a new strategy that computes the retraction only if a promising candidate for the step length is found. We further present a Riemannian Newton method with the new line-search strategy and prove its global convergence.

Optimization and Control

What problem does this paper attempt to address?

The paper attempts to address the problem of reducing computational cost in the optimization process on Riemannian manifolds, particularly during Armijo line search. Existing methods require the computation of retraction in the search direction at each iteration, which incurs high computational overhead. To this end, the authors propose a new line search method that reduces computational cost by computing retraction only when a suitable step size candidate is found. Specifically, the main contributions of the paper include: 1. **Proposing an improved Armijo line search method**: By introducing a new strategy that computes retraction only when a promising step size candidate is found, the computational cost is reduced. 2. **Combining the new line search strategy with the Riemannian Newton method**: A Riemannian Newton method based on the new line search strategy is proposed, and its global convergence is proven. 3. **Theoretical analysis and experimental validation**: The effectiveness and superiority of the new method are demonstrated through theoretical analysis and experimental validation. In summary, the paper aims to improve the efficiency and performance of optimization algorithms on Riemannian manifolds by improving the Armijo line search method.

Modified Armijo line-search in Riemannian optimization with reduced computational cost

The convergence of conjugate gradient method with nonmonotone line search

A class of nonmonotone Armijo-type line search method for unconstrained optimization

An Efficient Interior-Point Algorithm with New Non-Monotone Line Search Filter Method for Nonlinear Constrained Programming

Global convergence of Riemannian line search methods with a Zhang-Hager-type condition

Modified Line Search Sequential Quadratic Methods for Equality-Constrained Optimization with Unified Global and Local Convergence Guarantees

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

A new proximal heavy ball inexact line-search algorithm

Line Search Fixed Point Algorithms Based on Nonlinear Conjugate Gradient Directions: Application to Constrained Smooth Convex Optimization

A global and superlinearly convergent algorithm for nonlinear nondifferential convex programming problems with a generalized Armijo line-search

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

Determining Step Sizes in Geometric Optimization Algorithms.

Riemannian Interior Point Methods for Constrained Optimization on Manifolds

A Line Search Multigrid Method for Large-Scale Nonlinear Optimization

Riemannian Levenberg-Marquardt Method with Global and Local Convergence Properties

Two efficient nonlinear conjugate gradient methods for Riemannian manifolds

A More Lenient Stopping Rule for Line Search Algorithms

A modified Polak-Ribiere-Polyak type conjugate gradient method with two stepsize strategies for vector optimization

A New Retraction For Accelerating The Riemannian Three-Factor Low-Rank Matrix Completion Algorithm

Model Function Based Conditional Gradient Method with Armijo-like Line Search