Abstract:We study adversarial perturbations when the instances are uniformly distributed over $\{0,1\}^n$. We study both "inherent" bounds that apply to any problem and any classifier for such a problem as well as bounds that apply to specific problems and specific hypothesis classes. As the current literature contains multiple definitions of adversarial risk and robustness, we start by giving a taxonomy for these definitions based on their goals, we identify one of them as the one guaranteeing misclassification by pushing the instances to the error region. We then study some classic algorithms for learning monotone conjunctions and compare their adversarial risk and robustness under different definitions by attacking the hypotheses using instances drawn from the uniform distribution. We observe that sometimes these definitions lead to significantly different bounds. Thus, this study advocates for the use of the error-region definition, even though other definitions, in other contexts, may coincide with the error-region definition. Using the error-region definition of adversarial perturbations, we then study inherent bounds on risk and robustness of any classifier for any classification problem whose instances are uniformly distributed over $\{0,1\}^n$. Using the isoperimetric inequality for the Boolean hypercube, we show that for initial error $0.01$, there always exists an adversarial perturbation that changes $O(\sqrt{n})$ bits of the instances to increase the risk to $0.5$, making classifier's decisions meaningless. Furthermore, by also using the central limit theorem we show that when $n\to \infty$, at most $c \cdot \sqrt{n}$ bits of perturbations, for a universal constant $c< 1.17$, suffice for increasing the risk to $0.5$, and the same $c \cdot \sqrt{n} $ bits of perturbations on average suffice to increase the risk to $1$, hence bounding the robustness by $c \cdot \sqrt{n}$.

Robust Estimators in High Dimensions without the Computational Intractability

Robust Estimation and Shrinkage in Ultrahigh Dimensional Expectile Regression with Heavy Tails and Variance Heterogeneity

Robust Distribution Learning with Local and Global Adversarial Corruptions

Attacking Byzantine Robust Aggregation in High Dimensions

Robust Mean Estimation Without Moments for Symmetric Distributions

Beyond Moments: Robustly Learning Affine Transformations with Asymptotically Optimal Error

High Dimensional Distributed Gradient Descent with Arbitrary Number of Byzantine Attackers

Optimal Robust Estimation under Local and Global Corruptions: Stronger Adversary and Smaller Error

Robust Sparse Mean Estimation via Sum of Squares

High Dimensional Robust Sparse Regression

Robust Sparse Regression with Non-Isotropic Designs

Robust Sparse Estimation for Gaussians with Optimal Error under Huber Contamination

Optimal Robust Linear Regression in Nearly Linear Time

Beyond Catoni: Sharper Rates for Heavy-Tailed and Robust Mean Estimation

Covariance Estimation: Optimal Dimension-free Guarantees for Adversarial Corruption and Heavy Tails

A Geometric Unification of Distributionally Robust Covariance Estimators: Shrinking the Spectrum by Inflating the Ambiguity Set

Robust Estimation under the Wasserstein Distance

Robustness Implies Privacy in Statistical Estimation

Adversarial Risk and Robustness: General Definitions and Implications for the Uniform Distribution

Robust Distributed Learning Against Both Distributional Shifts and Byzantine Attacks

Distribution Learnability and Robustness