Abstract:In this work, we conduct a systematic study of stochastic saddle point problems (SSP) and stochastic variational inequalities (SVI) under the constraint of $(\epsilon,\delta)$-differential privacy (DP) in both Euclidean and non-Euclidean setups. We first consider Lipschitz convex-concave SSPs in the $\ell_p/\ell_q$ setup, $p,q\in[1,2]$. Here, we obtain a bound of $\tilde{O}\big(\frac{1}{\sqrt{n}} + \frac{\sqrt{d}}{n\epsilon}\big)$ on the strong SP-gap, where $n$ is the number of samples and $d$ is the dimension. This rate is nearly optimal for any $p,q\in[1,2]$. Without additional assumptions, such as smoothness or linearity requirements, prior work under DP has only obtained this rate when $p=q=2$ (i.e., only in the Euclidean setup). Further, existing algorithms have each only been shown to work for specific settings of $p$ and $q$ and under certain assumptions on the loss and the feasible set, whereas we provide a general algorithm for DP SSPs whenever $p,q\in[1,2]$. Our result is obtained via a novel analysis of the recursive regularization algorithm. In particular, we develop new tools for analyzing generalization, which may be of independent interest. Next, we turn our attention towards SVIs with a monotone, bounded and Lipschitz operator and consider $\ell_p$-setups, $p\in[1,2]$. Here, we provide the first analysis which obtains a bound on the strong VI-gap of $\tilde{O}\big(\frac{1}{\sqrt{n}} + \frac{\sqrt{d}}{n\epsilon}\big)$. For $p-1=\Omega(1)$, this rate is near optimal due to existing lower bounds. To obtain this result, we develop a modified version of recursive regularization. Our analysis builds on the techniques we develop for SSPs as well as employing additional novel components which handle difficulties arising from adapting the recursive regularization framework to SVIs.

Quantitative stability for stochastic tensor variational inequalities

Quantitative stability for a class of stochastic vector linear variational inequalities

Expected Residual Minimization Method for a Class of Stochastic Tensor Variational Inequalities

STOCHASTIC STABILITY OF QUASI-NON-INTEGRABLE-HAMILTONIAN SYSTEMS

Stochastic stabilization of quasi non-integrable Hamiltonian systems

Stability for a stochastic fractional differential variational inequality with Lévy jump

On quantitative stability in infinite-dimensional optimization under uncertainty

Quantitative Analysis for Perturbed Abstract Inequality Systems in Banach Spaces

Sample Average Approximation Method for a Class of Stochastic Variational Inequality Problems.

Generalized Smooth Stochastic Variational Inequalities: Almost Sure Convergence and Convergence Rates

Quantitative instability for stochastic scalar reaction-diffusion equations

Individual confidence intervals for true solutions to stochastic variational inequalities

Mean-square exponential stability of stochastic Volterra systems in infinite dimensions

Quantitative spectral stability for compact operators

Enforcing strong stability of explicit Runge--Kutta methods with superviscosity

First-order methods for Stochastic Variational Inequality problems with Function Constraints

Quantitative stability of Sobolev inequalities on compact Riemannian manifolds

On Multistage Pseudomonotone Stochastic Variational Inequalities

On stochastic fractional differential variational inequalities general system with Lévy jumps

Private Algorithms for Stochastic Saddle Points and Variational Inequalities: Beyond Euclidean Geometry

Nonconvex quasi-variational inequalities: stability analysis and application to numerical optimization