Abstract:In this paper, we focus on the relaxed proximal point algorithm (RPPA) for solving convex (possibly nonsmooth) optimization problems. We conduct a comprehensive study on three types of relaxation schedules: (i) constant schedule with relaxation parameter $\alpha_k\equiv \alpha \in (0, \sqrt{2}]$, (ii) the dynamic schedule put forward by Teboulle and Vaisbourd [TV23], and (iii) the silver stepsize schedule proposed by Altschuler and Parrilo [AP23b]. The latter two schedules were initially investigated for the gradient descent (GD) method and are extended to the RPPA in this paper. For type (i), we establish tight non-ergodic $O(1/N)$ convergence rate results measured by function value residual and subgradient norm, where $N$ denotes the iteration counter. For type (ii), we establish a convergence rate that is tight and approximately $\sqrt{2}$ times better than the constant schedule of type (i). For type (iii), aside from the original silver stepsize schedule put forward by Altschuler and Parrilo, we propose two new modified silver stepsize schedules, and for all the three silver stepsize schedules, $O(1/N^{1.2716})$ accelerated convergence rate results with respect to three different performance metrics are established. Furthermore, our research affirms the conjecture in [LG24][Conjecture 3.2] on GD method with the original silver stepsize schedule.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is related to **convergence analysis and acceleration performance of the Relaxed Proximal Point Algorithm (RPPA)**. Specifically, the author focuses on the performance of RPPA in solving convex optimization problems, especially the influence of different relaxation schedules on RPPA. The following are the main problems that this paper attempts to solve: 1. **Establishing tight complexity bounds**: For the constant relaxation parameter schedule $\alpha_k \equiv \alpha \in (0, \sqrt{2}]$, the author aims to establish tight non - ergodic $O(1/N)$ convergence rate results for the function - value residual and the sub - gradient norm. 2. **Improvement of dynamic relaxation parameter schedules**: For the dynamic relaxation parameter schedule proposed by Teboulle and Vaisbourd, the author establishes convergence rate results and proves that it is approximately $\sqrt{2}$ times faster than the constant relaxation parameter schedule. 3. **Extension and acceleration of silver step - size schedules**: The author introduces two new silver step - size schedules and proves that RPPA under all three silver step - size schedules can achieve an accelerated convergence rate of $O(1/N^{1.2716})$ under three different performance indicators. 4. **Verifying existing conjectures**: As a by - product, the author solves certain conjectures in the literature by providing formal proofs, such as the conjecture about the gradient descent method in [LG24, Conjecture 3.2]. ### Specific contributions of the paper 1. **Tight complexity bounds for constant relaxation parameter schedules**: - The author establishes tight non - ergodic $O(1/N)$ convergence rate results based on the function - value residual and the sub - gradient norm when $\alpha_k \equiv \alpha \in (0, \sqrt{2}]$ (see Theorem 3.1 and Remark 3.1). 2. **Improvement of dynamic relaxation parameter schedules**: - For the dynamic relaxation parameter schedule proposed by Teboulle and Vaisbourd, the author establishes tight complexity bounds and proves that it is about $\sqrt{2}$ times better than the result of the constant relaxation parameter schedule as $N \to \infty$ (see Theorem 4.1). 3. **Silver step - size schedules and their extensions**: - The author introduces two new silver step - size schedules and proves that RPPA under all three silver step - size schedules can achieve an accelerated convergence rate of $O(1/N^{1.2716})$ under three different performance indicators (see Theorem 5.3, Theorem 5.4 and Theorem 5.5). 4. **Convergence rate results of the gradient descent method**: - As a by - product of the research, the author establishes tight convergence rate results for the gradient descent method using silver step - size schedules in smooth convex optimization problems, verifying the conjecture in [LG24, Conjecture 3.2] (see Theorem 5.2). ### Conclusion By deeply analyzing the influence of different relaxation parameter schedules on RPPA, this paper not only establishes tight complexity bounds, but also proposes new step - size schedule schemes, significantly improving the convergence speed of RPPA. These results provide important theoretical support for the design and application of optimization algorithms.

Relaxed Proximal Point Algorithm: Tight Complexity Bounds and Acceleration without Momentum

A UNIFIED AND TIGHT LINEAR CONVERGENCE ANALYSIS OF THE RELAXED PROXIMAL POINT ALGORITHM

A Symplectic Discretization Based Proximal Point Algorithm for Convex Minimization

An adaptive proximal point algorithm framework and application to large-scale optimization

On the optimal linear convergence factor of the relaxed proximal point algorithm for monotone inclusion problems.

Accelerating Proximal Gradient Descent via Silver Stepsizes

Accelerated Quasi-Newton Proximal Extragradient: Faster Rate for Smooth Convex Optimization

Asynchronous Stochastic Proximal Methods for Nonconvex Nonsmooth Optimization.

A Simple Proximal Stochastic Gradient Method for Nonsmooth Nonconvex Optimization.

Accelerated Bregman Proximal Gradient Methods for Relatively Smooth Convex Optimization

New nonasymptotic convergence rates of stochastic proximal pointalgorithm for convex optimization problems

Accelerated Stochastic Algorithms for Nonconvex Finite-sum and Multi-block Optimization

A Bregman Proximal Stochastic Gradient Method with Extrapolation for Nonconvex Nonsmooth Problems

A Proximal-Gradient Method for Constrained Optimization

Proximal point type algorithms with relaxed and inertial effects beyond convexity

An optimal randomized incremental gradient method

Complexity of Inexact Proximal Point Algorithm for minimizing convex functions with Holderian Growth

On a Unified and Simplified Proof for the Ergodic Convergence Rates of PPM, PDHG and ADMM

Riemannian Proximal Gradient Methods (extended Version)

Simple and Optimal Stochastic Gradient Methods for Nonsmooth Nonconvex Optimization

Riemannian Stochastic Proximal Gradient Methods for Nonsmooth Optimization over the Stiefel Manifold