Abstract:Machine learning approaches relying on such criteria as adversarial robustness or multi-agent settings have raised the need for solving game-theoretic equilibrium problems. Of particular relevance to these applications are methods targeting finite-sum structure, which generically arises in empirical variants of learning problems in these contexts. Further, methods with computable approximation errors are highly desirable, as they provide verifiable exit criteria. Motivated by these applications, we study finite-sum monotone inclusion problems, which model broad classes of equilibrium problems. Our main contributions are variants of the classical Halpern iteration that employ variance reduction to obtain improved complexity guarantees in which $n$ component operators in the finite sum are ``on average'' either cocoercive or Lipschitz continuous and monotone, with parameter $L$. The resulting oracle complexity of our methods, which provide guarantees for the last iterate and for a (computable) operator norm residual, is $\widetilde{\mathcal{O}}( n + \sqrt{n}L\varepsilon^{-1})$, which improves upon existing methods by a factor up to $\sqrt{n}$. This constitutes the first variance reduction-type result for general finite-sum monotone inclusions and for more specific problems such as convex-concave optimization when operator norm residual is the optimality measure. We further argue that, up to poly-logarithmic factors, this complexity is unimprovable in the monotone Lipschitz setting; i.e., the provided result is near-optimal.

Stochastic forward-backward-half forward splitting algorithm with variance reduction

Forward-backward-half forward splitting algorithm with deviations

An outer reflected forward-backward splitting algorithm for solving monotone inclusions

A modification of the forward–backward splitting method for monotone inclusions

Splitting algorithms for solving state-dependent maximal monotone inclusion problems

A primal-dual backward reflected forward splitting algorithm for structured monotone inclusions

Almost-Surely Convergent Randomly Activated Monotone Operator Splitting Methods

A Relaxed Inertial Method for Solving Monotone Inclusion Problems with Applications

Forward–Reflected–Backward Splitting Algorithms with Momentum: Weak, Linear and Strong Convergence Results

Variance Reduced Halpern Iteration for Finite-Sum Monotone Inclusions

Variance reduced forward-reflected-backward algorithm for solving nonconvex finite-sum mixed variational inequalities

A new splitting method for solving composite monotone inclusions involving parallel-sum operators

New self-adaptive methods with double inertial steps for solving splitting monotone variational inclusion problems with applications

Strong Convergence of A Splitting Algorithm for Treating Monotone Operators

Parallel inertial forward–backward splitting methods for solving variational inequality problems with variational inclusion constraints

Three-operator reflected forward-backward splitting algorithm with double inertial effects

Stochastic Variance-Reduced Majorization-Minimization Algorithms

A fixed-time stable forward-backward dynamical system for solving generalized monotone inclusions

Iterative methods for solving monotone variational inclusions without prior knowledge of the Lipschitz constant of the single-valued operator

Solving Nonsmooth Nonconvex Compound Stochastic Programs with Applications to Risk Measure Minimization

Global Convergence of an Inexact Operator Splitting Method for Monotone Variational Inequalities