Abstract:Performance analysis of first-order algorithms with inexact oracle has gained recent attention due to various emerging applications in which obtaining exact gradients is impossible or computationally expensive. Previous research has demonstrated that the performance of accelerated first-order methods is more sensitive to gradient errors compared with non-accelerated ones. This paper investigates the nonasymptotic convergence bound of two accelerated methods with inexact gradients to solve deterministic smooth convex problems. Performance Estimation Problem (PEP) is used as the primary tool to analyze the convergence bounds of the underlying algorithms. By finding an analytical solution to PEP, we derive novel convergence bounds for Inexact Optimized Gradient Method (OGM) and Inexact Fast Gradient Method (FGM) with variable inexactness along iterations. Under the absolute error assumption, we derive bounds in which the accumulated errors are independent of the initial conditions and the trajectory of the sequences generated by the algorithms. Furthermore, we analyze the tradeoff between the convergence rate and accumulated error that guides finding the optimal stepsize. Finally, we determine the optimal strategy to set the gradient inexactness along iterations (if possible), ensuring that the accumulated error remains subordinate to the convergence rate.

Inexact and Implementable Accelerated Newton Proximal Extragradient Method for Convex Optimization

Inexact Proximal Gradient Methods for Non-convex and Non-smooth Optimization

Accelerated Quasi-Newton Proximal Extragradient: Faster Rate for Smooth Convex Optimization

Efficient Inexact Proximal Gradient Algorithm for Nonconvex Problems.

Nesterov's Acceleration For Approximate Newton.

Stochastic Newton Proximal Extragradient Method

Inexact Proximal Cubic Regularized Newton Methods for Convex Optimization

An Improved Inexact Newton Method

An Accelerated Distributed Method with Inexact Model of Relative Smoothness and Strong Convexity

Inexact Riemannian Gradient Descent Method for Nonconvex Optimization

Intermediate Gradient Methods with Relative Inexactness

An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization

Nonasymptotic Analysis of Accelerated Methods With Inexact Oracle Under Absolute Error Bound

An Accelerated Decentralized Stochastic Optimization Algorithm with Inexact Model

Inexact Proximal Gradient Algorithm with Random Reshuffling for Nonsmooth Optimization

A Relative Inexact Proximal Gradient Method With an Explicit Linesearch

An inexact regularized proximal Newton method without line search

Adaptive regularized quasi-newton method using inexact first-order information

Accelerated Proximal Gradient Methods for Nonconvex Programming