Abstract:An Adaptive Regularisation framework using Cubics (ARC) was proposed for unconstrained optimization and analysed in Cartis, Gould and Toint (Part I, Math Program, doi:10.1007/s10107-009-0286-5, 2009), generalizing at the same time an unpublished method due to Griewank (Technical Report NA/12, 1981, DAMTP, University of Cambridge), an algorithm by Nesterov and Polyak (Math Program 108(1):177–205, 2006) and a proposal by Weiser, Deuflhard and Erdmann (Optim Methods Softw 22(3):413–431, 2007). In this companion paper, we further the analysis by providing worst-case global iteration complexity bounds for ARC and a second-order variant to achieve approximate first-order, and for the latter second-order, criticality of the iterates. In particular, the second-order ARC algorithm requires at most \documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${\mathcal{O}(\epsilon^{-3/2})}$$\end{document} iterations, or equivalently, function- and gradient-evaluations, to drive the norm of the gradient of the objective below the desired accuracy \documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${\epsilon}$$\end{document}, and \documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${\mathcal{O}(\epsilon^{-3})}$$\end{document} iterations, to reach approximate nonnegative curvature in a subspace. The orders of these bounds match those proved for Algorithm 3.3 of Nesterov and Polyak which minimizes the cubic model globally on each iteration. Our approach is more general in that it allows the cubic model to be solved only approximately and may employ approximate Hessians.

Higher-Order Monte Carlo through Cubic Stratification

Linear Monte Carlo quadrature with optimal confidence intervals

Monte Carlo is a good sampling strategy for polynomial approximation in high dimensions

Achieving High Convergence Rates by Quasi-Monte Carlo and Importance Sampling for Unbounded Integrands

Monte Carlo Cubature Construction

Multilevel Monte Carlo with Numerical Smoothing for Robust and Efficient Computation of Probabilities and Densities

A Universal Median Quasi-Monte Carlo Integration

High-order numerical integration on self-affine sets

On high-order multilevel optimization strategies

Higher order scrambled digital nets achieve the optimal rate of the root mean square error for smooth integrands

Quasi-Monte Carlo for unbounded integrands with importance sampling

Quasi-Monte Carlo Beyond Hardy-Krause

Monte Carlo methods on compact complex manifolds using Bergman kernels

Optimal unbiased estimators via convex hulls

Multilevel higher order Quasi-Monte Carlo Bayesian Estimation

Speeding up Monte Carlo Integration: Control Neighbors for Optimal Convergence

The Trio Identity for Quasi-Monte Carlo Error

Higher order QMC Galerkin discretization for parametric operator equations

Volume Estimates for Singular sets and Critical Sets of Elliptic Equations with Hölder Coefficients

Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity