Abstract:We study adversarial perturbations when the instances are uniformly distributed over $\{0,1\}^n$. We study both "inherent" bounds that apply to any problem and any classifier for such a problem as well as bounds that apply to specific problems and specific hypothesis classes. As the current literature contains multiple definitions of adversarial risk and robustness, we start by giving a taxonomy for these definitions based on their goals, we identify one of them as the one guaranteeing misclassification by pushing the instances to the error region. We then study some classic algorithms for learning monotone conjunctions and compare their adversarial risk and robustness under different definitions by attacking the hypotheses using instances drawn from the uniform distribution. We observe that sometimes these definitions lead to significantly different bounds. Thus, this study advocates for the use of the error-region definition, even though other definitions, in other contexts, may coincide with the error-region definition. Using the error-region definition of adversarial perturbations, we then study inherent bounds on risk and robustness of any classifier for any classification problem whose instances are uniformly distributed over $\{0,1\}^n$. Using the isoperimetric inequality for the Boolean hypercube, we show that for initial error $0.01$, there always exists an adversarial perturbation that changes $O(\sqrt{n})$ bits of the instances to increase the risk to $0.5$, making classifier's decisions meaningless. Furthermore, by also using the central limit theorem we show that when $n\to \infty$, at most $c \cdot \sqrt{n}$ bits of perturbations, for a universal constant $c< 1.17$, suffice for increasing the risk to $0.5$, and the same $c \cdot \sqrt{n} $ bits of perturbations on average suffice to increase the risk to $1$, hence bounding the robustness by $c \cdot \sqrt{n}$.

A Notion of Uniqueness for the Adversarial Bayes Classifier

Adversarial Consistency and the Uniqueness of the Adversarial Bayes Classifier

On the Existence of the Adversarial Bayes Classifier (Extended Version)

Adversarial Risk and Robustness: General Definitions and Implications for the Uniform Distribution

The Consistency of Adversarial Training for Binary Classification

On the existence of solutions to adversarial training in multiclass classification

Structure of Classifier Boundaries: Case Study for a Naive Bayes Classifier

Adversarial Classification: Necessary conditions and geometric flows

Uniform Convergence of Adversarially Robust Classifiers

Asymptotic Behavior of Adversarial Training in Binary Classification

The Adversarial Consistency of Surrogate Risks for Binary Classification

The uniqueness of unconditional basis of the 2-convexified Tsirelson space, revisited

Generalized Bhattacharyya and Chernoff upper bounds on Bayes error using quasi-arithmetic means

Bayesian Adversarial Spheres: Bayesian Inference and Adversarial Examples in a Noiseless Setting

On existence, uniqueness and scalability of adversarial robustness measures for AI classifiers

Adversarially Robust One-class Novelty Detection

Local Competition and Uncertainty for Adversarial Robustness in Deep Learning

Genetic Column Generation for Computing Lower Bounds for Adversarial Classification

Intriguing Properties of Adversarial Examples

Universal Adversarial Examples and Perturbations for Quantum Classifiers

A PAC-Bayes Analysis of Adversarial Robustness