Abstract:SIAM Journal on Optimization, Volume 34, Issue 4, Page 3429-3455, December 2024. Inexact restoration (IR) methods are an important family of numerical methods for solving constrained optimization problems with applications to electronic structures and bilevel programming, among others areas. In these methods, the minimization is divided in two phases: decreasing infeasibility (feasibility phase) and improving optimality (optimality phase). The feasibility phase does not require the generated points to be feasible, so it has a practical appeal. In turn, the optimization phase involves minimizing a simplified model of the problem over a linearization of the feasible set. In this paper, we introduce a new optimization phase through a novel linearization that carries more information about complementarity than that employed in previous IR strategies. We then prove that the resulting algorithmic scheme is able to converge globally to the so-called complementary approximate KKT (CAKKT) points. This global convergence result improves upon all previous results for this class of methods. In particular, convergence to KKT points is established with the very weak CAKKT-regularity condition. Furthermore, to the best of our knowledge, this is the first time that a method for general nonlinear programming has reached CAKKT points without exogenous assumptions. From the practical point of view, the new optimization phase does not require significant additional computational effort compared to the usual one. Our theory also provides new insights, even for the classical IR method, for cases where it is reasonable to compute exact feasible points in the feasibility phase. We present numerical experiments on CUTEst problems to support our findings.

Inexact Restoration via random models for unconstrained noisy optimization

Inexact Restoration for Minimization with Inexact Evaluation both of the Objective Function and the Constraints

Improving the Global Convergence of Inexact Restoration Methods for Constrained Optimization Problems

Inexact Gauss-Newton methods with matrix approximation by sampling for nonlinear least-squares and systems

Adaptation to Inexactness for some Gradient-type Methods

Stochastic Optimization for Non-convex Problem with Inexact Hessian Matrix, Gradient, and Function

A Trust-Region Framework for Real-Time Optimization with Structural Process-Model Mismatch

Stochastic Approximation for Expectation Objective and Expectation Inequality-Constrained Nonconvex Optimization

Stochastic Optimization for Nonconvex Problem with Inexact Hessian Matrix, Gradient, and Function

Inexact model: a framework for optimization and variational inequalities

Optimal sampling for stochastic and natural gradient descent

Rockafellian Relaxation and Stochastic Optimization under Perturbations

Stochastic Second-order Methods for Non-convex Optimization with Inexact Hessian and Gradient

Inexact Proximal Gradient Algorithm with Random Reshuffling for Nonsmooth Optimization

Adaptive reconstruction of imperfectly-observed monotone functions, with applications to uncertainty quantification

A Gradient Smoothed Functional Algorithm with Truncated Cauchy Random Perturbations for Stochastic Optimization

Restarts subject to approximate sharpness: A parameter-free and optimal scheme for first-order methods

An Efficient Inexact Gauss–Seidel-Based Algorithm for Image Restoration with Mixed Noise

Reasons for the Superiority of Stochastic Estimators over Deterministic Ones: Robustness, Consistency and Perceptual Quality

A Stochastic Quasi-Newton Method in the Absence of Common Random Numbers

A Stochastic Operator Framework for Optimization and Learning with Sub-Weibull Errors