Abstract:In this paper, it was proposed a new concept of the inexact higher degree $(\delta, L, q)$-model of a function that is a generalization of the inexact $(\delta, L)$-model, $(\delta, L)$-oracle and $(\delta, L)$-oracle of degree $q \in [0,2)$. Some examples were provided to illustrate the proposed new model. Adaptive inexact gradient and fast gradient methods for convex and strongly convex functions were constructed and analyzed using the new proposed inexact model. A universal fast gradient method that allows solving optimization problems with a weaker level of smoothness, among them non-smooth problems was proposed. For convex optimization problems it was proved that the proposed gradient and fast gradient methods could be converged with rates $O\left(\frac{1}{k} + \frac{\delta}{k^{q/2}}\right)$ and $O\left(\frac{1}{k^2} + \frac{\delta}{k^{(3q-2)/2}}\right)$, respectively. For the gradient method, the coefficient of $\delta$ diminishes with $k$, and for the fast gradient method, there is no error accumulation for $q \geq 2/3$. It proposed a definition of an inexact higher degree oracle for strongly convex functions and a projected gradient method using this inexact oracle. For variational inequalities and saddle point problems, a higher degree inexact model and an adaptive method called Generalized Mirror Prox to solve such class of problems using the proposed inexact model were proposed. Some numerical experiments were conducted to demonstrate the effectiveness of the proposed inexact model, we test the universal fast gradient method to solve some non-smooth problems with a geometrical nature.

Higher degree inexact model for optimization problems

Higher Degree Inexact Model for Optimization problems

Proximal gradient methods with inexact oracle of degree q for composite optimization

Inexact model: a framework for optimization and variational inequalities

Adaptation to Inexactness for some Gradient-type Methods

Inexact Proximal Gradient Methods for Non-convex and Non-smooth Optimization

On quasi-convex smooth optimization problems by a comparison oracle

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

A new inexact gradient descent method with applications to nonsmooth convex optimization

Gradient Methods with Dynamic Inexact Oracles

Decentralized Constraint-Coupled Optimization with Inexact Oracle

Intermediate Gradient Methods with Relative Inexactness

Non-Euclidean High-Order Smooth Convex Optimization

Optimal inexactness schedules for tunable oracle-based methods

Optimal inexactness schedules for Tunable Oracle based Methods

On high-order multilevel optimization strategies

Inexact Riemannian Gradient Descent Method for Nonconvex Optimization

Inexact bilevel stochastic gradient methods for constrained and unconstrained lower-level problems

A Nonlinear Conjugate Gradient Method Using Inexact First-Order Information