Abstract:Despite considerable efforts on making them robust, real-world ML-based systems remain vulnerable to decision based attacks, as definitive proofs of their operational robustness have so far proven intractable. The canonical approach in robustness evaluation calls for adaptive attacks, that is with complete knowledge of the defense and tailored to bypass it. In this study, we introduce a more expansive notion of being adaptive and show how attacks but also defenses can benefit by it and by learning from each other through interaction. We propose and evaluate a framework for adaptively optimizing black-box attacks and defenses against each other through the competitive game they form. To reliably measure robustness, it is important to evaluate against realistic and worst-case attacks. We thus augment both attacks and the evasive arsenal at their disposal through adaptive control, and observe that the same can be done for defenses, before we evaluate them first apart and then jointly under a multi-agent perspective. We demonstrate that active defenses, which control how the system responds, are a necessary complement to model hardening when facing decision-based attacks; then how these defenses can be circumvented by adaptive attacks, only to finally elicit active and adaptive defenses. We validate our observations through a wide theoretical and empirical investigation to confirm that AI-enabled adversaries pose a considerable threat to black-box ML-based systems, rekindling the proverbial arms race where defenses have to be AI-enabled too. Succinctly, we address the challenges posed by adaptive adversaries and develop adaptive defenses, thereby laying out effective strategies in ensuring the robustness of ML-based systems deployed in the real-world.

Adversarial Robust Decision-Making under Uncertainty Learning and Dynamic Ensemble Selection

Dynamic ensemble selection based on Deep Neural Network Uncertainty Estimation for Adversarial Robustness

Attack As Defense: Characterizing Adversarial Examples Using Robustness.

Dynamic Defense Approach for Adversarial Robustness in Deep Neural Networks via Stochastic Ensemble Smoothed Model

Certifying Joint Adversarial Robustness for Model Ensembles

Improving Adversarial Robustness Via Promoting Ensemble Diversity

Improving Adversarial Robustness via Promoting Ensemble Diversity.

Diversity supporting robustness: Enhancing adversarial robustness via differentiated ensemble predictions

On the Certified Robustness for Ensemble Models and Beyond

Local Competition and Uncertainty for Adversarial Robustness in Deep Learning

Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks

Adversarial Markov Games: On Adaptive Decision-Based Attacks and Defenses

Revisiting Ensembles in an Adversarial Context: Improving Natural Accuracy

Synergy-of-Experts: Collaborate to Improve Adversarial Robustness

Adversarial robustness improvement for deep neural networks

ROBY: Evaluating the adversarial robustness of a deep model by its decision boundaries

A Unified Game-Theoretic Interpretation of Adversarial Robustness

Adversarial Defense Via Self-Orthogonal Randomization Super-Network.

CARE: Ensemble Adversarial Robustness Evaluation Against Adaptive Attackers for Security Applications

Ensemble Adversarial Defense via Integration of Multiple Dispersed Low Curvature Models

Robust Mode Connectivity-Oriented Adversarial Defense: Enhancing Neural Network Robustness Against Diversified $\ell_p$ Attacks