Abstract:Machine learning approaches relying on such criteria as adversarial robustness or multi-agent settings have raised the need for solving game-theoretic equilibrium problems. Of particular relevance to these applications are methods targeting finite-sum structure, which generically arises in empirical variants of learning problems in these contexts. Further, methods with computable approximation errors are highly desirable, as they provide verifiable exit criteria. Motivated by these applications, we study finite-sum monotone inclusion problems, which model broad classes of equilibrium problems. Our main contributions are variants of the classical Halpern iteration that employ variance reduction to obtain improved complexity guarantees in which $n$ component operators in the finite sum are ``on average'' either cocoercive or Lipschitz continuous and monotone, with parameter $L$. The resulting oracle complexity of our methods, which provide guarantees for the last iterate and for a (computable) operator norm residual, is $\widetilde{\mathcal{O}}( n + \sqrt{n}L\varepsilon^{-1})$, which improves upon existing methods by a factor up to $\sqrt{n}$. This constitutes the first variance reduction-type result for general finite-sum monotone inclusions and for more specific problems such as convex-concave optimization when operator norm residual is the optimality measure. We further argue that, up to poly-logarithmic factors, this complexity is unimprovable in the monotone Lipschitz setting; i.e., the provided result is near-optimal.

Stochastic Monotone Inclusion with Closed Loop Distributions

Invariant Closed Sets with Respect to Differential Inclusions with Time-Dependent Maximal Monotone Operators

Stochastic Proximal Point Methods for Monotone Inclusions under Expected Similarity

Locating Theorems of Differential Inclusions Governed by Maximally Monotone Operators

Stochastic Langevin Differential Inclusions with Applications to Machine Learning

Stabilization of differential inclusion systems with Markovian jumping

Density and co-density of the solution set of an evolution inclusion with maximal monotone operators

Constant Step Stochastic Approximations Involving Differential Inclusions: Stability, Long-Run Convergence and Applications

Continuous Dynamics Related to Monotone Inclusions and Non-Smooth Optimization Problems

A fixed-time stable forward-backward dynamical system for solving generalized monotone inclusions

Substochastic operators in symmetric spaces

Viability for locally monotone evolution inclusions and lower semicontinuous solutions of Hamilton-Jacobi-Bellman equations in infinite dimensions

Solution of Mismatched Monotone+Lipschitz Inclusion Problems

A parallel splitting method for weakly coupled monotone inclusions

On quantitative stability in infinite-dimensional optimization under uncertainty

Variance Reduced Halpern Iteration for Finite-Sum Monotone Inclusions

Well-Posedness and Convergence Results for History-Dependent Inclusions

Well-posedness of stochastic partial differential equations with fully local monotone coefficients

Discrete approximations and optimality conditions for integro-differential inclusions

Controllability of Nonlinear Second Order Neutral Stochastic Differential Inclusions with Infinite Delay

Stochastic optimization problems with nonlinear dependence on a probability measure via the Wasserstein metric