Abstract:Performance analysis of first-order algorithms with inexact oracle has gained recent attention due to various emerging applications in which obtaining exact gradients is impossible or computationally expensive. Previous research has demonstrated that the performance of accelerated first-order methods is more sensitive to gradient errors compared with non-accelerated ones. This paper investigates the nonasymptotic convergence bound of two accelerated methods with inexact gradients to solve deterministic smooth convex problems. Performance Estimation Problem (PEP) is used as the primary tool to analyze the convergence bounds of the underlying algorithms. By finding an analytical solution to PEP, we derive novel convergence bounds for Inexact Optimized Gradient Method (OGM) and Inexact Fast Gradient Method (FGM) with variable inexactness along iterations. Under the absolute error assumption, we derive bounds in which the accumulated errors are independent of the initial conditions and the trajectory of the sequences generated by the algorithms. Furthermore, we analyze the tradeoff between the convergence rate and accumulated error that guides finding the optimal stepsize. Finally, we determine the optimal strategy to set the gradient inexactness along iterations (if possible), ensuring that the accumulated error remains subordinate to the convergence rate.

Smale's α-theory for inexact Newton methods under the γ-condition

Smale’s point estimate theory for Newton’s method on Lie groups

Convergence Behavior of Gauss-Newton's Method and Extensions of the Smale Point Estimate Theory.

Newton's Method on Riemannian Manifolds: Smale's Point Estimate Theory under the Γ-Condition

Extended Newton Methods for Conic Inequalities: Approximate Solutions and the Extended Smale Α-Theory

Kantorovich-Type Convergence Criterion for Inexact Newton Methods

Local Convergence of Inexact Methods under the Hölder Condition

Newton's Method for Underdetermined Systems of Equations under the Γ-Condition

Newton's method on two step nilpotent lie groups: convergence criteria and the extended smale point estimate theory

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

On the Approximate Zero of Newton Method.

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

A Unified Convergence Theory for Newton-Type Methods for Zeros of Nonlinear Operators in Banach Spaces

CONVERGENCE CRITERION OF INEXACT METHODS FOR OPERATORS WITH H¨OLDER CONTINUOUS DERIVATIVES

Error Bounds for Approximate Solutions of Abstract Inequality Systems and Infinite Systems of Inequalities on Banach Spaces

Nonasymptotic Analysis of Accelerated Methods With Inexact Oracle Under Absolute Error Bound

An Uni ed Convergence Theory for Newton's Type Methods for Zeros of Nonlinear Operators in Banach spaces

Convergence analysis of the Gauss–Newton method for convex inclusion and convex-composite optimization problems

Convergence and Uniqueness Properties of Gauss-Newton's Method

Newton-type Methods for Inverse Singular Value Problems with Multiple Singular Values

Convergence Criterion of Newton's Method for Singular Systems with Constant Rank Derivatives