Abstract:Safety is a critical concern for the next generation of autonomy that is likely to rely heavily on deep neural networks for perception and control. Formally verifying the safety and robustness of well-trained DNNs and learning-enabled systems under attacks, model uncertainties, and sensing errors is essential for safe autonomy. This research proposes a framework to repair unsafe DNNs in safety-critical systems with reachability analysis. The repair process is inspired by adversarial training which has demonstrated high effectiveness in improving the safety and robustness of DNNs. Different from traditional adversarial training approaches where adversarial examples are utilized from random attacks and may not be representative of all unsafe behaviors, our repair process uses reachability analysis to compute the exact unsafe regions and identify sufficiently representative examples to enhance the efficacy and efficiency of the adversarial training. The performance of our framework is evaluated on two types of benchmarks without safe models as references. One is a DNN controller for aircraft collision avoidance with access to training data. The other is a rocket lander where our framework can be seamlessly integrated with the well-known deep deterministic policy gradient (DDPG) reinforcement learning algorithm. The experimental results show that our framework can successfully repair all instances on multiple safety specifications with negligible performance degradation. In addition, to increase the computational and memory efficiency of the reachability analysis algorithm, we propose a depth-first-search algorithm that combines an existing exact analysis method with an over-approximation approach based on a new set representation. Experimental results show that our method achieves a five-fold improvement in runtime and a two-fold improvement in memory usage compared to exact analysis.

NeVer 2.0: Learning, Verification and Repair of Deep Neural Networks

VeRe: Verification Guided Synthesis for Repairing Deep Neural Networks.

Interpretability Based Neural Network Repair

Improving Neural Network Verification through Spurious Region Guided Refinement.

NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems

DelBugV: Delta-Debugging Neural Network Verifiers

Automated Verification of Neural Networks: Advances, Challenges and Perspectives

ADVREPAIR:Provable Repair of Adversarial Attack

Online Verification of Deep Neural Networks under Domain Shift or Network Updates

Safety Analysis of Deep Neural Networks

Neural Network Repair with Reachability Analysis

DNNV: A Framework for Deep Neural Network Verification

Neuro-Symbolic Verification of Deep Neural Networks

Automatic Fault Detection for Deep Learning Programs Using Graph Transformations

SynapNet: A Complementary Learning System Inspired Algorithm With Real-Time Application in Multimodal Perception

DeepDyve: Dynamic Verification for Deep Neural Networks

Repairing Deep Neural Networks Based on Behavior Imitation

CLEVEREST: Accelerating CEGAR-based Neural Network Verification via Adversarial Attacks

Architecture-Preserving Provable Repair of Deep Neural Networks

Vehicle: Bridging the Embedding Gap in the Verification of Neuro-Symbolic Programs

VNN: Verification-Friendly Neural Networks with Hard Robustness Guarantees