Abstract:This paper settles an open and challenging question pertaining to the design of simple and optimal high-order methods for solving smooth and monotone variational inequalities (VIs). A VI involves finding $x^\star \in \mathcal{X}$ such that $\langle F(x), x - x^\star\rangle \geq 0$ for all $x \in \mathcal{X}$. We consider the setting in which $F$ is smooth with up to $(p-1)^{th}$-order derivatives. For $p = 2$, the cubic regularized Newton method was extended to VIs with a global rate of $O(\epsilon^{-1})$. An improved rate of $O(\epsilon^{-2/3}\log\log(1/\epsilon))$ can be obtained via an alternative second-order method, but this method requires a nontrivial line-search procedure as an inner loop. Similarly, high-order methods based on line-search procedures have been shown to achieve a rate of $O(\epsilon^{-2/(p+1)}\log\log(1/\epsilon))$. As emphasized by Nesterov, however, such procedures do not necessarily imply practical applicability in large-scale applications, and it would be desirable to complement these results with a simple high-order VI method that retains the optimality of the more complex methods. We propose a $p^{th}$-order method that does \textit{not} require any line search procedure and provably converges to a weak solution at a rate of $O(\epsilon^{-2/(p+1)})$. We prove that our $p^{th}$-order method is optimal in the monotone setting by establishing a matching lower bound under a generalized linear span assumption. Our method with restarting attains a linear rate for smooth and uniformly monotone VIs and a local superlinear rate for smooth and strongly monotone VIs. Our method also achieves a global rate of $O(\epsilon^{-2/p})$ for solving smooth and nonmonotone VIs satisfying the Minty condition and when augmented with restarting it attains a global linear and local superlinear rate for smooth and nonmonotone VIs satisfying the uniform/strong Minty condition.

An Optimal Method for High-Order Mixed Derivatives of Bivariate Functions

An optimal method for high order mixed derivatives of bivariate functions

About optimization of methods for mixed derivatives of bivariate functions

On optimal recovering high order partial derivatives of bivariate functions

Stable Lagrangian numerical differentiation with the highest order of approximation

On Optimal Recovery and Information Complexity in Numerical Differentiation and Summation

Numerical Differentiation for the Second Order Derivatives of Functions of Two Variables

Exact estimates of high-order derivatives in Sobolev spaces

Reconstruction of High Order Derivatives from Input Data

The Discrete Adjoint Method: Efficient Derivatives for Functions of Discrete Sequences

Stability of Multiquadric Quasi-Interpolation to Approximate High Order Derivatives

Construction of multivariate polynomial approximation kernels via semidefinite programming

An Optimal Fourth Order Derivative-Free Numerical Algorithm for Multiple Roots

An optimal interpolation set for model-based derivative-free optimization methods

Optimal estimators of cross-partial derivatives and surrogates of functions

Non-Euclidean High-Order Smooth Convex Optimization

Perseus: A Simple and Optimal High-Order Method for Variational Inequalities

High order numerical methods for solving high orders functional differential equations

First Derivative Approximations and Applications

Optimization-Aided Construction of Multivariate Chebyshev Polynomials