Abstract:Verifying safety of neural network control systems that use images as input is a difficult problem because, from a given system state, there is no known way to mathematically model what images are possible in the real-world. We build on recent work that considers a surrogate verification approach, training a conditional generative adversarial network (cGAN) as an image generator in place of the real world. This enables set-based formal analysis of the closed-loop system, providing analysis beyond simulation and testing. While existing work is effective on small examples, excessive overapproximation both within a single control period and across multiple control periods limits its scalability. We propose approaches to overcome these two sources of error. First, we overcome one-step error by composing the system's dynamics along with the cGAN and neural network controller, without losing the dependencies between input states and the control outputs as in the monotonic analysis of the system dynamics. Second, we reduce multi-step error by repeating the single-step composition, essentially unrolling multiple steps of the control loop into a large neural network. We then leverage existing network verification tools to compute accurate reachable sets for multiple steps, avoiding the accumulation of abstraction error at each step. We demonstrate the effectiveness of our approach in terms of both accuracy and scalability using two case studies: an autonomous aircraft taxiing system and an advanced emergency braking system. On the aircraft taxiing system, the converged reachable set is 175% larger using the prior baseline method compared with our proposed approach. On the emergency braking system, with 24x the number of image output variables from the cGAN, the baseline method fails to prove any states are safe, whereas our improvements enable set-based safety analysis.

Towards Safety Verification of Direct Perception Neural Networks

Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions

Verifiable Obstacle Detection

Creating a Formally Verified Neural Network for Autonomous Navigation: An Experience Report

Towards Robust Direct Perception Networks for Automated Driving

Toward Scalable Verification for Safety-Critical Deep Networks

Verification and Design of Robust and Safe Neural Network-enabled Autonomous Systems

Neuro-Symbolic Verification of Deep Neural Networks

SDLV: Verification of Steering Angle Safety for Self-Driving Cars

Assumption Generation for the Verification of Learning-Enabled Autonomous Systems

Generate and Verify: Semantically Meaningful Formal Analysis of Neural Network Perception Systems

Scalable Surrogate Verification of Image-based Neural Network Control Systems using Composition and Unrolling

Perception Simplex: Verifiable Collision Avoidance in Autonomous Vehicles Amidst Obstacle Detection Faults

A Dual Approach to Scalable Verification of Deep Networks

CAMUS: A Framework to Build Formal Specifications for Deep Perception Systems Using Simulators

Compositional Verification for Autonomous Systems with Deep Learning Components

Data-Driven Modeling and Verification of Perception-Based Autonomous Systems

Tackling Occlusions & Limited Sensor Range with Set-based Safety Verification

Real-Time Evaluation of Perception Uncertainty and Validity Verification of Autonomous Driving

Closed-loop Analysis of Vision-based Autonomous Systems: A Case Study