Abstract:The Performance Estimation Problem methodology makes it possible to determine the exact worst-case performance of an optimization method. In this work, we generalize this framework to first-order methods involving linear operators. This extension requires an explicit formulation of interpolation conditions for those linear operators. We consider the class of linear operators $\mathcal{M}:x \mapsto Mx$ where matrix $M$ has bounded singular values, and the class of linear operators where $M$ is symmetric and has bounded eigenvalues. We describe interpolation conditions for these classes, i.e. necessary and sufficient conditions that, given a list of pairs $\{(x_i,y_i)\}$, characterize the existence of a linear operator mapping $x_i$ to $y_i$ for all $i$. Using these conditions, we first identify the exact worst-case behavior of the gradient method applied to the composed objective $h\circ \mathcal{M}$, and observe that it always corresponds to $\mathcal{M}$ being a scaling operator. We then investigate the Chambolle-Pock method applied to $f+g\circ \mathcal{M}$, and improve the existing analysis to obtain a proof of the exact convergence rate of the primal-dual gap. In addition, we study how this method behaves on Lipschitz convex functions, and obtain a numerical convergence rate for the primal accuracy of the last iterate. We also show numerically that averaging iterates is beneficial in this setting.

Error estimates for simplified Levenberg–Marquardt method for nonlinear ill-posed operator equations in Hilbert Spaces

On inertial Levenberg-Marquardt type methods for solving nonlinear ill-posed operator equations

Local Convergence of Inexact Methods under the Hölder Condition

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Stochastic Data-Driven Bouligand Landweber Method for Solving Non-smooth Inverse Problems

An Analysis of a Multi-Level Projected Steepest Descent Iteration for Nonlinear Inverse Problems in Banach Spaces Subject to Stability Constraints

An extended study of two multistep higher order convergent methods for solving nonlinear equations

An asymptotical regularization with convex constraints for inverse problems

Nonconforming Finite Element Discretisation for Semilinear Problems with Trilinear Nonlinearity

Stable gradient projection method for nonlinear conditionally well-posed inverse problems

A modified iteratively regularized Landweber iteration method: Hölder stability and convergence rates

The Levenberg–Marquardt method: an overview of modern convergence theories and more

A New Adaptive Levenberg–Marquardt Method for Nonlinear Equations and Its Convergence Rate under the Hölderian Local Error Bound Condition

A fresh look at nonsmooth Levenberg–Marquardt methods with applications to bilevel optimization

Linear methods for non-linear inverse problems

An Enhanced Levenberg--Marquardt Method via Gram Reduction

Modified iterated Tikhonov methods for solving systems of nonlinear ill-posed equations

Stopping criteria for nonlinear variational problems: iterative approach

A rational conjugate gradient method for linear ill-conditioned problems

Levenberg-Marquardt method with Singular Scaling and applications

Interpolation Conditions for Linear Operators and Applications to Performance Estimation Problems