Abstract:Neural networks (NNs) are increasingly applied in safety-critical systems such as autonomous vehicles. However, they are fragile and are often ill-behaved. Consequently, their behaviors should undergo rigorous guarantees before deployment in practice. In this paper, we propose a set-boundary reachability method to investigate the safety verification problem of NNs from topological perspectives. Given an NN with an input set and a safe set, the safety verification problem is to determine whether all outputs of the NN resulting from the input set fall within the safe set. In our method, the homeomorphism property of NNs is first exploited, which establishes rigorous guarantees between the boundaries of the input set and the boundaries of the output set. A homeomorphism is a special case of open maps, and consequently our set-boundary method is considered to be generalized to situations with open map property then 2 2. The exploitation of these two properties facilitates reachability computations via extracting subsets of the input set rather than the entire input set, thus controlling the wrapping effect in reachability analysis and facilitating the reduction of computation burdens for safety verification. The homeomorphism property exists in some widely used NNs such as invertible residual networks (i-ResNets) and Neural ordinary differential equations (Neural ODEs), and the open map is a less strict topological property and is easier to satisfy compared with homeomorphisms. For NNs establishing either of these two properties, our set-boundary reachability method only needs to perform reachability analysis on the boundary of the input set. Moreover, for NNs that do not feature these properties with respect to the input set, we also explore subsets of the input set for establishing the local homeomorphism property and then abandon these subsets for reachability computations. Finally, some examples demonstrate the performance of our proposed method.

Safety Analysis of Deep Neural Networks

There is Limited Correlation Between Coverage and Robustness for Deep Neural Networks

Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety

Introduction to Neural Network Verification

DeepDyve: Dynamic Verification for Deep Neural Networks

Automated Verification of Neural Networks: Advances, Challenges and Perspectives

Analyzing Deep Neural Networks with Symbolic Propagation: Towards Higher Precision and Faster Verification

Model-Agnostic Reachability Analysis on Deep Neural Networks

Enumerating Safe Regions in Deep Neural Networks with Provable Probabilistic Guarantees

On Evaluating Neural Network Backdoor Defenses

Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction and Clustering

Verifying Safety of Neural Networks from Topological Perspectives

An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences

Verifying Global Two-Safety Properties in Neural Networks with Confidence

Analyzing neural network behavior through deep statistical model checking

SoK: Certified Robustness for Deep Neural Networks

Quantifying safety risks of deep neural networks

Toward Scalable Verification for Safety-Critical Deep Networks

Secure deep learning engineering: A road towards quality assurance of intelligent systems

Evaluating Deep Neural Networks in Deployment (A Comparative and Replicability Study)

Verifying the Generalization of Deep Learning to Out-of-Distribution Domains