Abstract:We present an adaptive trust-region method for unconstrained optimization that allows inexact solutions to the trust-region subproblems. Our method is a simple variant of the classical trust-region method of Sorensen [1]. The method achieves the best possible convergence bound up to an additive log factor, for finding an $\epsilon$-approximate stationary point, i.e., $O( \Delta_f L^{1/2} \epsilon^{-3/2}) + \tilde{O}(1)$ iterations where $L$ is the Lipschitz constant of the Hessian, $\Delta_f$ is the optimality gap, and $\epsilon$ is the termination tolerance for the gradient norm. This improves over existing trust-region methods whose worst-case bound is at least a factor of $L$ worse. We compare our performance with state-of-the-art trust-region (TRU) and cubic regularization (ARC) methods from the GALAHAD library on the CUTEst benchmark set on problems with more than 100 variables. We use fewer function, gradient, and Hessian evaluations than these methods. For instance, our algorithm's median number of gradient evaluations is $23$ compared to $36$ for TRU and $29$ for ARC. Compared to the conference version of this paper [2], our revised method includes practical enhancements and a refined subproblems termination criterion. These modifications dramatically improved performance, including an order of magnitude reduction in the shifted geometric mean of wall-clock times.

Improved Trust-Region Derivative-Free Algorithm for General Constrained Optimization Problems

Improved Algorithms Based on Trust Region Framework for Solving Unconstrained Derivative Free Optimization Problems

Globally Convergent Composite-Step Trust-Region Framework for Model-Based Real-Time Optimization

On Convergence Analysis of a Derivative-Free Trust Region Algorithm for Constrained Optimization with Separable Structure

An improved nonmonotone adaptive trust region method

An Improved Algorithm of Trust Region Method

A robust trust region method for general constrained optimization

An ODE-based Trust Region Method for Unconstrained Optimization Problems.

A New Trust Region Algorithm for Unconstrained Optimization

A Derivative-Free Trust-Region Algorithm for Reliability-Based Optimization

A Hybrid Trust Region Algorithm for Unconstrained Optimization

A Trust-Region Framework for Real-Time Optimization with Structural Process-Model Mismatch

An Improved Non-Monotonic Adaptive Trust Region Algorithm for Unconstrained Optimization

A Trust Region Algorithm with Conjugate Gradient Technique for Optimization Problems

A Trust Region Algorithm for Constrained Nonsmooth Optimization Problems

A simple and practical adaptive trust-region method

A TRUST-REGION METHOD FOR NONSMOOTH NONCONVEX OPTIMIZATION

A globally convergent composite‐step trust‐region framework for real‐time optimization

A robust trust region method for nonlinear optimization with inequality constraint

An adaptive simple model trust region algorithm based on new weak secant equations

One Method of Solving Constrained Optimization Problem Basing on the Trust-Region