Abstract:Deep neural networks have become widely used, obtaining remarkable results in domains such as computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, and bio-informatics, where they have produced results comparable to human experts. However, these networks can be easily fooled by adversarial perturbations: minimal changes to correctly-classified inputs, that cause the network to mis-classify them. This phenomenon represents a concern for both safety and security, but it is currently unclear how to measure a network's robustness against such perturbations. Existing techniques are limited to checking robustness around a few individual input points, providing only very limited guarantees. We propose a novel approach for automatically identifying safe regions of the input space, within which the network is robust against adversarial perturbations. The approach is data-guided, relying on clustering to identify well-defined geometric regions as candidate safe regions. We then utilize verification techniques to confirm that these regions are safe or to provide counter-examples showing that they are not safe. We also introduce the notion of targeted robustness which, for a given target label and region, ensures that a NN does not map any input in the region to the target label. We evaluated our technique on the MNIST dataset and on a neural network implementation of a controller for the next-generation Airborne Collision Avoidance System for unmanned aircraft (ACAS Xu). For these networks, our approach identified multiple regions which were completely safe as well as some which were only safe for specific labels. It also discovered several adversarial perturbations of interest.

DeepCDCL: An CDCL-based Neural Network Verification Framework

CLEVEREST: Accelerating CEGAR-based Neural Network Verification via Adversarial Attacks

A DPLL(T) Framework for Verifying Deep Neural Networks

NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems

DeepDyve: Dynamic Verification for Deep Neural Networks

Verifying Neural Network Controlled Systems Using Neural Networks

Robustness Verification of Classification Deep Neural Networks Via Linear Programming.

Conv-Reluplex : A Verification Framework For Convolution Neural Networks (S).

DeepSafe: A Data-driven Approach for Checking Adversarial Robustness in Neural Networks

Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models

COCL: an Intelligent Framework for Enhancing Deep Learning-Based Vulnerability Detection

DeepGlobal - A Global Robustness Verifiable FNN Framework.

DeepGlobal: A Framework for Global Robustness Verification of Feedforward Neural Networks

DCLane for Real-time Understanding of Lane Markings.

DeepAuto: A First Step Towards Formal Verification of Deep Learning Systems (S).

Analyzing neural network behavior through deep statistical model checking

Online Verification of Deep Neural Networks under Domain Shift or Network Updates

Deep Collaborative Learning for Visual Recognition.

Simultaneous Synthesis and Verification of Neural Control Barrier Functions through Branch-and-Bound Verification-in-the-loop Training

Toward Scalable Verification for Safety-Critical Deep Networks

Harnessing Neuron Stability to Improve DNN Verification