Abstract:Recent advances in Deep Neural Networks (DNNs) have led to the development of DNN-driven autonomous cars that, using sensors like camera, LiDAR, etc., can drive without any human intervention. Most major manufacturers including Tesla, GM, Ford, BMW, and Waymo/Google are working on building and testing different types of autonomous vehicles. The lawmakers of several US states including California, Texas, and New York have passed new legislation to fast-track the process of testing and deployment of autonomous vehicles on their roads. However, despite their spectacular progress, DNNs, just like traditional software, often demonstrate incorrect or unexpected corner case behaviors that can lead to potentially fatal collisions. Several such real-world accidents involving autonomous cars have already happened including one which resulted in a fatality. Most existing testing techniques for DNN-driven vehicles are heavily dependent on the manual collection of test data under different driving conditions which become prohibitively expensive as the number of test conditions increases. In this paper, we design, implement and evaluate DeepTest, a systematic testing tool for automatically detecting erroneous behaviors of DNN-driven vehicles that can potentially lead to fatal crashes. First, our tool is designed to automatically generated test cases leveraging real-world changes in driving conditions like rain, fog, lighting conditions, etc. DeepTest systematically explores different parts of the DNN logic by generating test inputs that maximize the numbers of activated neurons. DeepTest found thousands of erroneous behaviors under different realistic driving conditions (e.g., blurring, rain, fog, etc.) many of which lead to potentially fatal crashes in three top performing DNNs in the Udacity self-driving car challenge.

DeepState

DeepState: Selecting Test Suites to Enhance the Robustness of Recurrent Neural Networks

There is Limited Correlation Between Coverage and Robustness for Deep Neural Networks

An Empirical Study on Correlation between Coverage and Robustness for Deep Neural Networks

Adaptive deep learning for network intrusion detection by risk analysis

An Intrusion Detection System Using a Deep Neural Network with Gated Recurrent Units.

DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars

Comparative Study Of Cnn And Rnn For Deep Learning Based Intrusion Detection System

DeepGini: prioritizing massive tests to enhance the robustness of deep neural networks

Excitement Surfeited Turns to Errors: Deep Learning Testing Framework Based on Excitable Neurons

Neuron Sensitivity Guided Test Case Selection for Deep Learning Testing

DeepStellar: Model-based quantitative analysis of stateful deep learning systems.(2019)

DeepIA: An Interpretability Analysis based Test Data Generation Method for DNN

Spiking Deep Residual Networks.

Adaptive Test Selection for Deep Neural Networks

DeepStellar: Model-Based Quantitative Analysis of Stateful Deep Learning Systems

DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks

Rnntcs: A Test Case Selection Method for Recurrent Neural Networks

Deep-Net: Deep Neural Network for Cyber Security Use Cases

A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks