Abstract:High-order tensor methods that employ Taylor-based local models of positive integer degree $p$ within adaptive regularization frameworks have recently garnered significant research interest for both convex and nonconvex optimization problems. The well-known $p$th-order adaptive regularization (AR$p$) method has demonstrated optimal worst-case global convergence rates and local convergence rates. At each iteration of this method, a local model of the objective function, consisting of a $p$th-order Taylor approximation and a $(p+1)$th-order regularization term, is minimized. However, for $p \geq 2$, the subproblem typically contains multiple local minima. It is recommended to solve the subproblem globally when $p = 2$, as the global minimizer is guaranteed to be a descent direction. However, for any $p \geq 3$, it remains an open question how to efficiently minimize the subproblem and which minimizer to select. In this context, designing efficient AR$p$ variants is a critical area of interest, especially for $p \geq 3$. In this paper, we extend the interpolation-based updating strategies, originally introduced by Gould, Porcelli, and Toint in 2012 for $p = 2$, to cases where $p \geq 3$. This updating strategy enables the regularization parameter to adjust adaptively based on interpolation models. We propose a novel prerejection mechanism that rejects unwanted subproblem minimizers prior to any function evaluations, thereby reducing computational costs for $p\geq 3$. Numerical results illustrate how the proposed interpolation-based updating strategy and the prerejection mechanism for $p = 3$ enhance the overall numerical performance.

A Unified Adaptive Tensor Approximation Scheme to Accelerate Composite Convex Optimization.

A Unified Scheme to Accelerate Adaptive Cubic Regularization and Gradient Methods for Convex Optimization

An Optimal High-Order Tensor Method for Convex Optimization

Towards Unified Acceleration of High-Order Algorithms under H\"{o}lder Continuity and Uniform Convexity

Unified Acceleration of High-Order Algorithms under General Hölder Continuity

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

A Unified Algorithmic Framework for Distributed Composite Optimization.

Efficient Implementation of Third-order Tensor Methods with Adaptive Regularization for Unconstrained Optimization

Accelerating Adaptive Cubic Regularization of Newton's Method Via Random Sampling

A FISTA-type accelerated gradient algorithm for solving smooth nonconvex composite optimization problems

Adaptively Accelerating Cubic Regularized Newton's Methods for Convex Optimization via Random Sampling

A Stochastic Nesterov’s Smoothing Accelerated Method for General Nonsmooth Constrained Stochastic Composite Convex Optimization

Implementable tensor methods in unconstrained convex optimization

An Accelerated Regularized Chebyshev-Halley Method for Unconstrained Optimization

Nesterov's Acceleration For Approximate Newton.

On Adaptive Cubic Regularized Newton's Methods for Convex Optimization via Random Sampling

A Distributed Nesterov-Like Gradient Tracking Algorithm for Composite Constrained Optimization

OPTAMI: Global Superlinear Convergence of High-order Methods

Adaptive Proximal Average Approximation for Composite Convex Minimization.

Acceleration via symplectic discretization of high-resolution differential equations

Adaptive smoothing mini-batch stochastic accelerated gradient method for nonsmooth convex stochastic composite optimization