Abstract:Deep neural networks have become widely used, obtaining remarkable results in domains such as computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, and bio-informatics, where they have produced results comparable to human experts. However, these networks can be easily fooled by adversarial perturbations: minimal changes to correctly-classified inputs, that cause the network to mis-classify them. This phenomenon represents a concern for both safety and security, but it is currently unclear how to measure a network's robustness against such perturbations. Existing techniques are limited to checking robustness around a few individual input points, providing only very limited guarantees. We propose a novel approach for automatically identifying safe regions of the input space, within which the network is robust against adversarial perturbations. The approach is data-guided, relying on clustering to identify well-defined geometric regions as candidate safe regions. We then utilize verification techniques to confirm that these regions are safe or to provide counter-examples showing that they are not safe. We also introduce the notion of targeted robustness which, for a given target label and region, ensures that a NN does not map any input in the region to the target label. We evaluated our technique on the MNIST dataset and on a neural network implementation of a controller for the next-generation Airborne Collision Avoidance System for unmanned aircraft (ACAS Xu). For these networks, our approach identified multiple regions which were completely safe as well as some which were only safe for specific labels. It also discovered several adversarial perturbations of interest.

Measuring Robustness of Deep Neural Networks from the Lens of Statistical Model Checking.

There is Limited Correlation Between Coverage and Robustness for Deep Neural Networks

Certifying Semantic Robustness of Deep Neural Networks

An Empirical Study on Correlation between Coverage and Robustness for Deep Neural Networks

Guiding the Comparison of Neural Network Local Robustness: an Empirical Study

SoK: Certified Robustness for Deep Neural Networks

Certifying Global Robustness for Deep Neural Networks

ROBY: Evaluating the adversarial robustness of a deep model by its decision boundaries

Robustness Verification of Classification Deep Neural Networks Via Linear Programming.

Measuring Neural Net Robustness with Constraints

Towards Precise Observations of Neural Model Robustness in Classification

Globally-Robust Neural Networks

Robustness of Neural Networks: A Probabilistic and Practical Approach

Predicted Robustness as QoS for Deep Neural Network Models

Analyzing neural network behavior through deep statistical model checking

A Survey of Neural Network Robustness Assessment in Image Recognition

Global Robustness Verification Networks

Tight Verification of Probabilistic Robustness in Bayesian Neural Networks

Fast Robustness Prediction for Deep Neural Network

DeepSafe: A Data-driven Approach for Checking Adversarial Robustness in Neural Networks

Certifying Robustness of Convolutional Neural Networks with Tight Linear Approximation