Abstract:We present an adaptive trust-region method for unconstrained optimization that allows inexact solutions to the trust-region subproblems. Our method is a simple variant of the classical trust-region method of Sorensen [1]. The method achieves the best possible convergence bound up to an additive log factor, for finding an $\epsilon$-approximate stationary point, i.e., $O( \Delta_f L^{1/2} \epsilon^{-3/2}) + \tilde{O}(1)$ iterations where $L$ is the Lipschitz constant of the Hessian, $\Delta_f$ is the optimality gap, and $\epsilon$ is the termination tolerance for the gradient norm. This improves over existing trust-region methods whose worst-case bound is at least a factor of $L$ worse. We compare our performance with state-of-the-art trust-region (TRU) and cubic regularization (ARC) methods from the GALAHAD library on the CUTEst benchmark set on problems with more than 100 variables. We use fewer function, gradient, and Hessian evaluations than these methods. For instance, our algorithm's median number of gradient evaluations is $23$ compared to $36$ for TRU and $29$ for ARC. Compared to the conference version of this paper [2], our revised method includes practical enhancements and a refined subproblems termination criterion. These modifications dramatically improved performance, including an order of magnitude reduction in the shifted geometric mean of wall-clock times.

Beyond Nonconvexity: A Universal Trust-Region Method with New Analyses

A Trust Region-Type Method for Solving Monotone Variational Inequality

A TRUST-REGION METHOD FOR NONSMOOTH NONCONVEX OPTIMIZATION

A simple and practical adaptive trust-region method

A nonmonotone trust region method for unconstrained optimization

The Convergence of the Generalized Lanczos Trust-Region Method for the Trust-Region Subproblem

A trust region-type normal map-based semismooth Newton method for nonsmooth nonconvex composite optimization

An interior-point trust-region method for nonsmooth regularized bound-constrained optimization

Globally Convergent Composite-Step Trust-Region Framework for Model-Based Real-Time Optimization

On globally solving nonconvex trust region subproblem via projected gradient method

An Improved Non-Monotonic Adaptive Trust Region Algorithm for Unconstrained Optimization

Convergence and worst-case complexity of adaptive Riemannian trust-region methods for optimization on manifolds

A new nonmonotone trust-region method of conic model for solving unconstrained optimization

Hölderian Error Bounds and Kurdyka-Łojasiewicz Inequality for the Trust Region Subproblem

Local convergence analysis of an inexact trust-region method for nonsmooth optimization

An adaptive simple model trust region algorithm based on new weak secant equations

A Trust-Region Algorithm for Noisy Equality Constrained Optimization

A new trust region algorithm for nonsmooth convex minimization

A New Trust-Region Constraint Method for Sequential Convex Programming Framework for Entry Guidance

Effective nonmonotone trust region method based on a simple cubic model for unconstrained optimization problems

Two trust region type algorithms for solving nonconvex-strongly concave minimax problems