Abstract:This paper settles an open and challenging question pertaining to the design of simple and optimal high-order methods for solving smooth and monotone variational inequalities (VIs). A VI involves finding $x^\star \in \mathcal{X}$ such that $\langle F(x), x - x^\star\rangle \geq 0$ for all $x \in \mathcal{X}$. We consider the setting in which $F$ is smooth with up to $(p-1)^{th}$-order derivatives. For $p = 2$, the cubic regularized Newton method was extended to VIs with a global rate of $O(\epsilon^{-1})$. An improved rate of $O(\epsilon^{-2/3}\log\log(1/\epsilon))$ can be obtained via an alternative second-order method, but this method requires a nontrivial line-search procedure as an inner loop. Similarly, high-order methods based on line-search procedures have been shown to achieve a rate of $O(\epsilon^{-2/(p+1)}\log\log(1/\epsilon))$. As emphasized by Nesterov, however, such procedures do not necessarily imply practical applicability in large-scale applications, and it would be desirable to complement these results with a simple high-order VI method that retains the optimality of the more complex methods. We propose a $p^{th}$-order method that does \textit{not} require any line search procedure and provably converges to a weak solution at a rate of $O(\epsilon^{-2/(p+1)})$. We prove that our $p^{th}$-order method is optimal in the monotone setting by establishing a matching lower bound under a generalized linear span assumption. Our method with restarting attains a linear rate for smooth and uniformly monotone VIs and a local superlinear rate for smooth and strongly monotone VIs. Our method also achieves a global rate of $O(\epsilon^{-2/p})$ for solving smooth and nonmonotone VIs satisfying the Minty condition and when augmented with restarting it attains a global linear and local superlinear rate for smooth and nonmonotone VIs satisfying the uniform/strong Minty condition.

A Block Coordinate and Variance-Reduced Method for Generalized Variational Inequalities of Minty Type

A Trust Region-Type Method for Solving Monotone Variational Inequality

Variance-reduction for Variational Inequality Problems with Bregman Distance Function

Beyond Monotone Variational Inequalities: Solution Methods and Iteration Complexities

Method with batching for stochastic finite-sum variational inequalities in non-Euclidean setting

Adaptive Methods or Variational Inequalities with Relatively Smooth and Reletively Strongly Monotone Operators

Blockwise Stochastic Variance-Reduced Methods with Parallel Speedup for Multi-Block Bilevel Optimization

A New Alternating Direction Method for Solving Variational Inequalities

A New Decomposition Method for Variational Inequalities with Linear Constraints

Some Adaptive First-order Methods for Variational Inequalities with Relatively Strongly Monotone Operators and Generalized Smoothness

Adaptive Methods for Variational Inequalities under Relaxed Smoothness Assumption

Two Novel Algorithms for Solving Variational Inequality Problems Governed by Fixed Point Problems and Their Applications

Perseus: A Simple and Optimal High-Order Method for Variational Inequalities

Optimal Analysis of Method with Batching for Monotone Stochastic Finite-Sum Variational Inequalities

New inertial forward-backward type for variational inequalities with Quasi-monotonicity

Global Convergence of an Inexact Operator Splitting Method for Monotone Variational Inequalities

Variance reduced forward-reflected-backward algorithm for solving nonconvex finite-sum mixed variational inequalities

Algorithms for solving variational inequalities and saddle point problems with some generalizations of Lipschitz property for operators

Methods for Solving Variational Inequalities with Markovian Stochasticity

On the convergence properties of non-Euclidean extragradient methods for variational inequalities with generalized monotone operators