Abstract:A significant milestone in modern gradient-based optimization was achieved with the development of Nesterov's accelerated gradient descent (NAG) method. This forward-backward technique has been further advanced with the introduction of its proximal generalization, commonly known as the fast iterative shrinkage-thresholding algorithm (FISTA), which enjoys widespread application in image science and engineering. Nonetheless, it remains unclear whether both NAG and FISTA exhibit linear convergence for strongly convex functions. Remarkably, these algorithms demonstrate convergence without requiring any prior knowledge of strongly convex modulus, and this intriguing characteristic has been acknowledged as an open problem in the comprehensive review [Chambolle and Pock, 2016, Appendix B]. In this paper, we address this question by utilizing the high-resolution ordinary differential equation (ODE) framework. Expanding upon the established phase-space representation, we emphasize the distinctive approach employed in crafting the Lyapunov function, which involves a dynamically adapting coefficient of kinetic energy that evolves throughout the iterations. Furthermore, we highlight that the linear convergence of both NAG and FISTA is independent of the parameter $r$. Additionally, we demonstrate that the square of the proximal subgradient norm likewise advances towards linear convergence.

Forward-Backward algorithms for weakly convex problems

Forward-Backward Algorithm for Functions with Locally Lipschitz Gradient: Applications to Mean Field Games

Stochastic proximal gradient methods for nonconvex problems in Hilbert spaces

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

Convergence rate of inexact proximal point algorithms for operator with holder metric subregularity

Local convergence of an inexact proximal algorithm for weakly convex functions

Convergence of a class of algorithms with the level-set subdifferential error bound

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Proximal Algorithms for a class of abstract convex functions

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

A Mirror Inertial Forward–Reflected–Backward Splitting: Convergence Analysis Beyond Convexity and Lipschitz Smoothness

Convergence of inertial prox-penalization and inertial forward–backward algorithms for solving monotone bilevel equilibrium problems

Strongly convergent inertial forward-backward-forward algorithm without on-line rule for variational inequalities

Accelerated forward–backward algorithms for structured monotone inclusions

Incremental Methods for Weakly Convex Optimization

New strong convergence analysis for variational inequalities and fixed point problems

Variance reduced forward-reflected-backward algorithm for solving nonconvex finite-sum mixed variational inequalities

Convergence results for proximal point algorithm with inertial and correction terms

Linear convergence of forward-backward accelerated algorithms without knowledge of the modulus of strong convexity

Primal-dual algorithm for weakly convex functions under sharpness conditions

Modified proximal gradient methods involving double inertial extrapolations for monotone inclusion