Abstract:An Adaptive Regularisation framework using Cubics (ARC) was proposed for unconstrained optimization and analysed in Cartis, Gould and Toint (Part I, Math Program, doi:10.1007/s10107-009-0286-5, 2009), generalizing at the same time an unpublished method due to Griewank (Technical Report NA/12, 1981, DAMTP, University of Cambridge), an algorithm by Nesterov and Polyak (Math Program 108(1):177–205, 2006) and a proposal by Weiser, Deuflhard and Erdmann (Optim Methods Softw 22(3):413–431, 2007). In this companion paper, we further the analysis by providing worst-case global iteration complexity bounds for ARC and a second-order variant to achieve approximate first-order, and for the latter second-order, criticality of the iterates. In particular, the second-order ARC algorithm requires at most \documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${\mathcal{O}(\epsilon^{-3/2})}$$\end{document} iterations, or equivalently, function- and gradient-evaluations, to drive the norm of the gradient of the objective below the desired accuracy \documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${\epsilon}$$\end{document}, and \documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${\mathcal{O}(\epsilon^{-3})}$$\end{document} iterations, to reach approximate nonnegative curvature in a subspace. The orders of these bounds match those proved for Algorithm 3.3 of Nesterov and Polyak which minimizes the cubic model globally on each iteration. Our approach is more general in that it allows the cubic model to be solved only approximately and may employ approximate Hessians.

An adaptive cubic regularisation algorithm based on interior point methods for solving nonlinear inequality constrained optimization

A sequential adaptive regularisation using cubics algorithm for solving nonlinear equality constrained optimization

A line search filter sequential adaptive cubic regularisation algorithm for nonlinearly constrained optimization

An Improved Smoothing Technique-Based Control Vector Parameterization Method for Optimal Control Problems with Inequality Path Constraints

A Novel Penalty Approach for Nonlinear Dynamic Optimization Problems with Inequality Path Constraints

Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity

Structured Regularization In Barrier Nlp For Optimization Models With Dependent Constraints

Barrier NLP Methods with Structured Regularization for Optimization of Degenerate Optimization Problems

Regularized methods via cubic model subspace minimization for nonconvex optimization

A Momentum Accelerated Adaptive Cubic Regularization Method for Nonconvex Optimization

Regularized Interior Point Methods for Constrained Optimization and Control

Convergence depth control for interior point methods

An interior-point trust-region method for nonsmooth regularized bound-constrained optimization

Cubic-quartic regularization models for solving polynomial subproblems in third-order tensor methods

A modified adaptive cubic regularization method for large-scale unconstrained optimization problem

An Infeasible Interior-point Arc-search Algorithm for Nonlinear Constrained Optimization

The Regularization Continuation Method for Optimization Problems with Nonlinear Equality Constraints

A penalty barrier framework for nonconvex constrained optimization

A conic regularized Barzilai–Borwein trust region method for large scale unconstrained optimization

A Note on Inexact Condition for Cubic Regularized Newton's Method

A computationally efficient arc-search interior-point algorithm for nonlinear constrained optimization