Abstract:In this work, we conduct a systematic study of stochastic saddle point problems (SSP) and stochastic variational inequalities (SVI) under the constraint of $(\epsilon,\delta)$-differential privacy (DP) in both Euclidean and non-Euclidean setups. We first consider Lipschitz convex-concave SSPs in the $\ell_p/\ell_q$ setup, $p,q\in[1,2]$. Here, we obtain a bound of $\tilde{O}\big(\frac{1}{\sqrt{n}} + \frac{\sqrt{d}}{n\epsilon}\big)$ on the strong SP-gap, where $n$ is the number of samples and $d$ is the dimension. This rate is nearly optimal for any $p,q\in[1,2]$. Without additional assumptions, such as smoothness or linearity requirements, prior work under DP has only obtained this rate when $p=q=2$ (i.e., only in the Euclidean setup). Further, existing algorithms have each only been shown to work for specific settings of $p$ and $q$ and under certain assumptions on the loss and the feasible set, whereas we provide a general algorithm for DP SSPs whenever $p,q\in[1,2]$. Our result is obtained via a novel analysis of the recursive regularization algorithm. In particular, we develop new tools for analyzing generalization, which may be of independent interest. Next, we turn our attention towards SVIs with a monotone, bounded and Lipschitz operator and consider $\ell_p$-setups, $p\in[1,2]$. Here, we provide the first analysis which obtains a bound on the strong VI-gap of $\tilde{O}\big(\frac{1}{\sqrt{n}} + \frac{\sqrt{d}}{n\epsilon}\big)$. For $p-1=\Omega(1)$, this rate is near optimal due to existing lower bounds. To obtain this result, we develop a modified version of recursive regularization. Our analysis builds on the techniques we develop for SSPs as well as employing additional novel components which handle difficulties arising from adapting the recursive regularization framework to SVIs.

Gluing Dynamics: $\varepsilon$-Precision in Solving a Non-Archimedean Inverse Problem

Landen transformations applied to approximation

Some remarks on approximation in several complex variables

Approximations of Functions With Essential Singularities with Applications to Painlevé's First Transcendent

Geometric and arithmetic aspects of approximation vectors

Computation of polynomial and rational approximations in complex domains by the -method

Exploiting locality in sparse polynomial approximation of parametric elliptic PDEs and application to parameterized domains

Rational Points Near Manifolds, Homogeneous Dynamics, and Oscillatory Integrals

Superconvergent Non-Polynomial Approximations

Inexpensive polynomial-degree-robust equilibrated flux a posteriori estimates for isogeometric analysis

New methods for quasi-interpolation approximations: resolution of odd-degree singularities

Regular polynomial interpolation and approximation of global solutions of linear partial differential equations

Adelic perturbation of rational functions and applications

Private Algorithms for Stochastic Saddle Points and Variational Inequalities: Beyond Euclidean Geometry

Dynamics of Non-polar Solutions to the Discrete Painlevé I Equation

Functional A Posteriori Error Equalities for Conforming Mixed Approximations of Elliptic Problems

A Geometric Approach to Polynomial and Rational Approximation

The repulsion property in nonlinear elasticity and a numerical scheme to circumvent it

Automatic rational approximation and linearization of nonlinear eigenvalue problems

Exact a posteriori error control for variational problems via convex duality and explicit flux reconstruction