Abstract:In this paper, we focus on the relaxed proximal point algorithm (RPPA) for solving convex (possibly nonsmooth) optimization problems. We conduct a comprehensive study on three types of relaxation schedules: (i) constant schedule with relaxation parameter $\alpha_k\equiv \alpha \in (0, \sqrt{2}]$, (ii) the dynamic schedule put forward by Teboulle and Vaisbourd [TV23], and (iii) the silver stepsize schedule proposed by Altschuler and Parrilo [AP23b]. The latter two schedules were initially investigated for the gradient descent (GD) method and are extended to the RPPA in this paper. For type (i), we establish tight non-ergodic $O(1/N)$ convergence rate results measured by function value residual and subgradient norm, where $N$ denotes the iteration counter. For type (ii), we establish a convergence rate that is tight and approximately $\sqrt{2}$ times better than the constant schedule of type (i). For type (iii), aside from the original silver stepsize schedule put forward by Altschuler and Parrilo, we propose two new modified silver stepsize schedules, and for all the three silver stepsize schedules, $O(1/N^{1.2716})$ accelerated convergence rate results with respect to three different performance metrics are established. Furthermore, our research affirms the conjecture in [LG24][Conjecture 3.2] on GD method with the original silver stepsize schedule.

Proximal point type algorithms with relaxed and inertial effects beyond convexity

Relaxed-Inertial Proximal Point Algorithms for Nonconvex Equilibrium Problems with Applications

Proximal Point Algorithms with Inertial Extrapolation for Quasi-convex Pseudo-monotone Equilibrium Problems

An extension of the proximal point algorithm beyond convexity

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

Proximal-point-like algorithms for abstract convex minimisation problems

Proximal Point Algorithms for Nonsmooth Convex Optimization with Fixed Point Constraints

Convergence results for proximal point algorithm with inertial and correction terms

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Proximal Algorithms for a class of abstract convex functions

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

A modified inertial proximal minimization algorithm for structured nonconvex and nonsmooth problem

An implementable proximal-type method for computing critical points to minimization problems with a nonsmooth and nonconvex constraint

A strongly convergent inertial inexact proximal-point algorithm for monotone inclusions with applications to variational inequalities

On the optimal linear convergence factor of the relaxed proximal point algorithm for monotone inclusion problems.

An interior proximal gradient method for nonconvex optimization

Relaxed Proximal Point Algorithm: Tight Complexity Bounds and Acceleration without Momentum

A Two-Step Proximal Point Algorithm for Nonconvex Equilibrium Problems with Applications to Fractional Programming

Local convergence of an inexact proximal algorithm for weakly convex functions

A Symplectic Discretization Based Proximal Point Algorithm for Convex Minimization

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds