Abstract:In this paper we develop accelerated first-order methods for convex optimization with locally Lipschitz continuous gradient (LLCG), which is beyond the well-studied class of convex optimization with Lipschitz continuous gradient. In particular, we first consider unconstrained convex optimization with LLCG and propose accelerated proximal gradient (APG) methods for solving it. The proposed APG methods are equipped with a verifiable termination criterion and enjoy an operation complexity of ${\cal O}(\varepsilon^{-1/2}\log \varepsilon^{-1})$ and ${\cal O}(\log \varepsilon^{-1})$ for finding an $\varepsilon$-residual solution of an unconstrained convex and strongly convex optimization problem, respectively. We then consider constrained convex optimization with LLCG and propose an first-order proximal augmented Lagrangian method for solving it by applying one of our proposed APG methods to approximately solve a sequence of proximal augmented Lagrangian subproblems. The resulting method is equipped with a verifiable termination criterion and enjoys an operation complexity of ${\cal O}(\varepsilon^{-1}\log \varepsilon^{-1})$ and ${\cal O}(\varepsilon^{-1/2}\log \varepsilon^{-1})$ for finding an $\varepsilon$-KKT solution of a constrained convex and strongly convex optimization problem, respectively. All the proposed methods in this paper are parameter-free or almost parameter-free except that the knowledge on convexity parameter is required. In addition, preliminary numerical results are presented to demonstrate the performance of our proposed methods. To the best of our knowledge, no prior studies were conducted to investigate accelerated first-order methods with complexity guarantees for convex optimization with LLCG. All the complexity results obtained in this paper are new.

Accelerated First-order Methods for Geodesically Convex Optimization on Riemannian Manifolds.

Towards Riemannian Accelerated Gradient Methods

Accelerated Riemannian Optimization: Handling Constraints with a Prox to Bound Geometric Penalties

An Accelerated Distributed Method with Inexact Model of Relative Smoothness and Strong Convexity

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

Accelerated first-order methods for convex optimization with locally Lipschitz continuous gradient

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

Faster First-Order Methods for Stochastic Non-Convex Optimization on Riemannian Manifolds

First-order Methods for Geodesically Convex Optimization

Faster Acceleration for Steepest Descent

Accelerating Nesterov's Method for Strongly Convex Functions with Lipschitz Gradient

Riemannian Proximal Gradient Methods (extended Version)

Geometry of First-Order Methods and Adaptive Acceleration.

Accelerated optimization algorithms and ordinary differential equations: the convex non Euclidean case

Geometry of First-Order Methods and Adaptive Acceleration

Inexact Riemannian Gradient Descent Method for Nonconvex Optimization

Riemannian proximal gradient methods

The Global R-linear Convergence of Nesterov's Accelerated Gradient Method with Unknown Strongly Convex Parameter

Riemannian Acceleration with Preconditioning for symmetric eigenvalue problems

Gauges and Accelerated Optimization over Smooth and/or Strongly Convex Sets

Accelerated First-Order Optimization under Nonlinear Constraints