Abstract:Deep Neural Networks (DNNs) are widely used for computer vision tasks. However, it has been shown that deep models are vulnerable to adversarial attacks, i.e. , their performances drop when imperceptible perturbations are made to the original inputs, which may further degrade the following visual tasks or introduce new problems such as data and privacy security. Hence, metrics for evaluating the robustness of deep models against adversarial attacks are desired. However, previous metrics are mainly proposed for evaluating the adversarial robustness of shallow networks on the small-scale datasets. Although the Cross Lipschitz Extreme Value for nEtwork Robustness (CLEVER) metric has been proposed for large-scale datasets ( e.g. , the ImageNet dataset), it is computationally expensive and its performance relies on a tractable number of samples. In this paper, we propose the Adversarial Converging Time Score (ACTS), an attack-dependent metric that quantifies the adversarial robustness of a DNN on a specific input. Our key observation is that local neighborhoods on a DNN’s output surface would have different shapes given different inputs. Hence, given different inputs, it requires different time for converging to an adversarial sample. Based on this geometry meaning, ACTS measures the converging time as an adversarial robustness metric. We validate the effectiveness and generalization of the proposed ACTS metric against different adversarial attacks on the large-scale ImageNet dataset using state-of-the-art deep networks. Extensive experiments show that our ACTS metric is an efficient and effective adversarial metric over the previous CLEVER metric.

CLEVEREST: Accelerating CEGAR-based Neural Network Verification via Adversarial Attacks

VeRe: Verification Guided Synthesis for Repairing Deep Neural Networks.

Online Verification of Deep Neural Networks under Domain Shift or Network Updates

Empirial study on security verification and assessment of neural network accelerator

DeepDyve: Dynamic Verification for Deep Neural Networks

DelBugV: Delta-Debugging Neural Network Verifiers

Set-Based Training for Neural Network Verification

Model-Agnostic Reachability Analysis on Deep Neural Networks

Neuro-Symbolic Verification of Deep Neural Networks

Harnessing Neuron Stability to Improve DNN Verification

Global Robustness Verification Networks

NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems

Bound Tightening using Rolling-Horizon Decomposition for Neural Network Verification

SoK: Certified Robustness for Deep Neural Networks

DeepGlobal: A Framework for Global Robustness Verification of Feedforward Neural Networks

Efficient verification of neural networks based on neuron branching and LP abstraction

Generate and Verify: Semantically Meaningful Formal Analysis of Neural Network Perception Systems

Formal Verification of Deep Neural Networks for Object Detection

A Dual Approach to Scalable Verification of Deep Networks

A Geometrical Approach to Evaluate the Adversarial Robustness of Deep Neural Networks

AdversarialStyle: GAN Based Style Guided Verification Framework for Deep Learning Systems