Abstract:Autonomous systems are increasingly implemented using end-to-end learning-based controllers. Such controllers make decisions that are executed on the real system, with images as one of the primary sensing modalities. Deep neural networks form a fundamental building block of such controllers. Unfortunately, the existing neural-network verification tools do not scale to inputs with thousands of dimensions -- especially when the individual inputs (such as pixels) are devoid of clear physical meaning. This paper takes a step towards connecting exhaustive closed-loop verification with high-dimensional controllers. Our key insight is that the behavior of a high-dimensional controller can be approximated with several low-dimensional controllers. To balance the approximation accuracy and verifiability of our low-dimensional controllers, we leverage the latest verification-aware knowledge distillation. Then, we inflate low-dimensional reachability results with statistical approximation errors, yielding a high-confidence reachability guarantee for the high-dimensional controller. We investigate two inflation techniques -- based on trajectories and control actions -- both of which show convincing performance in three OpenAI gym benchmarks.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to provide reliable safety guarantees in autonomous systems controlled by high - dimensional controllers (HDCs). Specifically, the paper focuses on how to verify end - to - end deep neural network controllers with high - dimensional inputs such as images to ensure the safety of these controllers in practical applications. This problem becomes particularly complex because existing neural network verification tools cannot be directly applied to systems with thousands of - dimensional inputs, especially when individual inputs (such as pixels) lack clear physical meanings. ### Main contributions of the paper 1. **Two verification methods**: A method combining reachability analysis and statistical inference is proposed to provide safety guarantees for systems controlled by neural networks with thousands of input dimensions. 2. **New neural network approximation technique**: A technique for training multiple low - dimensional controllers (LDCs) is developed. These controllers can collectively imitate the behavior of high - dimensional controllers and reduce over - approximation errors. 3. **Implementation and evaluation**: The above - mentioned verification methods are implemented in three popular OpenAI Gym case studies, including the inverted pendulum, mountain car, and cart - pole tasks. ### Key steps of the solution 1. **Training low - dimensional controllers**: - Use the knowledge distillation technique to train multiple low - dimensional controllers (LDCs) from the high - dimensional controller (HDC). These LDCs can operate on dynamic states instead of directly processing images. - Optimize the LDCs through the two - objective gradient descent method, minimizing the mean - squared error (MSE) and the Lipschitz constant simultaneously to improve the verifiability and predictability of the LDCs. 2. **Reachability analysis of low - dimensional systems**: - Perform reachability analysis on each LDC to generate over - approximate reachability tubes. - Use the POLAR toolbox for reachability analysis. This toolbox approximates activation functions by calculating univariate Bernstein polynomials and uses Taylor or Bernstein polynomials for tight over - approximation. 3. **Calculating statistical difference bounds**: - Define two types of difference functions: trajectory - based difference \(\beta\) and action - based difference \(\gamma\). - Use the conformal prediction method to calculate the statistical upper bounds of these differences, ensuring that the differences between the LDC and the HDC are statistically bounded at a given confidence level. 4. **Extending reachability tubes**: - Extend the reachability tubes of the low - dimensional system according to the statistical difference bounds to include the reachability tubes of the high - dimensional system. - Propose two extension methods, trajectory - based and action - based, to ensure that the extended reachability tubes can contain the real HDC reachability tubes with high probability. 5. **Iterative retraining and remeshing**: - For the situation where the HDC simulation is successful but the safety verification fails in the current initial set sub - region, perform iterative retraining and remeshing to reduce over - approximation errors and conform to the prediction difference bounds. ### Conclusion Through the above methods, the paper successfully provides reliable safety guarantees for systems controlled by high - dimensional controllers, especially in application scenarios with high - dimensional inputs such as images. This provides new ideas and technical support for future safety verification in critical applications such as self - driving cars and aircraft landing guidance.

Bridging Dimensions: Confident Reachability for High-Dimensional Controllers

Scalable Synthesis of Verified Controllers in Deep Reinforcement Learning

Safe and Reliable Training of Learning-Based Aerospace Controllers

Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions

Interval Reachability of Nonlinear Dynamical Systems with Neural Network Controllers

Enhancing Safety and Robustness of Vision-Based Controllers via Reachability Analysis

Discovering Closed-Loop Failures of Vision-Based Controllers via Reachability Analysis

Model-Free Verification for Neural Network Controlled Systems

Simultaneous Synthesis and Verification of Neural Control Barrier Functions through Branch-and-Bound Verification-in-the-loop Training

Scalable Surrogate Verification of Image-based Neural Network Control Systems using Composition and Unrolling

Reachability Analysis and Safety Verification for Neural Network Control Systems

Enhancing the Performance of DeepReach on High-Dimensional Systems through Optimizing Activation Functions

Hybrid Controller Synthesis for Nonlinear Systems Subject to Reach-Avoid Constraints.

Verified Safe Reinforcement Learning for Neural Network Dynamic Models

Verification of Neural Reachable Tubes via Scenario Optimization and Conformal Prediction

Generating Formal Safety Assurances for High-Dimensional Reachability

Efficient Verification of Control Systems with Neural Network Controllers.

A General Framework for Verification and Control of Dynamical Models via Certificate Synthesis

Reachability Verification Based Reliability Assessment for Deep Reinforcement Learning Controlled Robotics and Autonomous Systems

A General Verification Framework for Dynamical and Control Models via Certificate Synthesis

Distributionally Robust Statistical Verification with Imprecise Neural Networks