Abstract:In this work we propose a general nonmonotone line-search method for nonconvex multi\-objective optimization problems with convex constraints. At the $k$th iteration, the degree of nonmonotonicity is controlled by a vector $\nu_{k}$ with nonnegative components. Different choices for $\nu_{k}$ lead to different nonmonotone step-size rules. Assuming that the sequence $\left\{\nu_{k}\right\}_{k\geq 0}$ is summable, and that the $i$th objective function has Hölder continuous gradient with smoothness parameter $\theta_i \in(0,1]$, we show that the proposed method takes no more than $\mathcal{O}\left(\epsilon^{-\left(1+\frac{1}{\theta_{\min}}\right)}\right)$ iterations to find a $\epsilon$-approximate Pareto critical point for a problem with $m$ objectives and $\theta_{\min}= \min_{i=1,\dots, m} \{\theta_i\}$. In particular, this complexity bound applies to the methods proposed by Drummond and Iusem (Comput. Optim. Appl. 28: 5--29, 2004), by Fazzio and Schuverdt (Optim. Lett. 13: 1365--1379, 2019), and by Mita, Fukuda and Yamashita (J. Glob. Optim. 75: 63--90, 2019). The generality of our approach also allows the development of new methods for multiobjective optimization. As an example, we propose a new nonmonotone step-size rule inspired by the Metropolis criterion. Preliminary numerical results illustrate the benefit of nonmonotone line searches and suggest that our new rule is particularly suitable for multiobjective problems in which at least one of the objectives has many non-global local minimizers.

Universal heavy-ball method for nonconvex optimization under Hölder continuous Hessians

Parameter-free accelerated gradient descent for nonconvex minimization

Primitive Heavy-ball Dynamics Achieves $O(\varepsilon^{-7/4})$ Convergence for Nonconvex Optimization

Newton-CG methods for nonconvex unconstrained optimization with Hölder continuous Hessian

Efficient Lipschitzian Global Optimization of Hölder Continuous Multivariate Functions

On Optimal Universal First-Order Methods for Minimizing Heterogeneous Sums

Heavy Ball Momentum for Non-Strongly Convex Optimization

A Universal Accelerated Primal–Dual Method for Convex Optimization Problems

First and zeroth-order implementations of the regularized Newton method with lazy approximated Hessians

Super-Universal Regularized Newton Method

Adaptive proximal gradient methods are universal without approximation

Unified Acceleration of High-Order Algorithms under General Hölder Continuity

Towards Unified Acceleration of High-Order Algorithms under H\"{o}lder Continuity and Uniform Convexity

Heavy-ball Algorithms Always Escape Saddle Points

Accelerated Over-Relaxation Heavy-Ball Methods with Provable Acceleration and Global Convergence

Gradient-Free Methods for Non-Smooth Convex Stochastic Optimization with Heavy-Tailed Noise on Convex Compact

Restarted Nonconvex Accelerated Gradient Descent: No More Polylogarithmic Factor in the O(<SUP>-7/4</SUP>) Complexity

An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes

Universal Gradient Methods for Stochastic Convex Optimization

Universal nonmonotone line search method for nonconvex multiobjective optimization problems with convex constraints