Abstract:Recent advances in Deep Neural Networks (DNNs) have led to the development of DNN-driven autonomous cars that, using sensors like camera, LiDAR, etc., can drive without any human intervention. Most major manufacturers including Tesla, GM, Ford, BMW, and Waymo/Google are working on building and testing different types of autonomous vehicles. The lawmakers of several US states including California, Texas, and New York have passed new legislation to fast-track the process of testing and deployment of autonomous vehicles on their roads. However, despite their spectacular progress, DNNs, just like traditional software, often demonstrate incorrect or unexpected corner case behaviors that can lead to potentially fatal collisions. Several such real-world accidents involving autonomous cars have already happened including one which resulted in a fatality. Most existing testing techniques for DNN-driven vehicles are heavily dependent on the manual collection of test data under different driving conditions which become prohibitively expensive as the number of test conditions increases. In this paper, we design, implement and evaluate DeepTest, a systematic testing tool for automatically detecting erroneous behaviors of DNN-driven vehicles that can potentially lead to fatal crashes. First, our tool is designed to automatically generated test cases leveraging real-world changes in driving conditions like rain, fog, lighting conditions, etc. DeepTest systematically explores different parts of the DNN logic by generating test inputs that maximize the numbers of activated neurons. DeepTest found thousands of erroneous behaviors under different realistic driving conditions (e.g., blurring, rain, fog, etc.) many of which lead to potentially fatal crashes in three top performing DNNs in the Udacity self-driving car challenge.

Research on an Acceleration Test Method of Automatic Driving Based on Deep Neural Network

A Deep Learning Method for Lane Changing Situation Assessment and Decision Making.

Driving Behaviour Style Study with a Hybrid Deep Learning Framework Based on GPS Data

DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars

A system of vision sensor based deep neural networks for complex driving scene analysis in support of crash risk assessment and prevention

Personalized Driver Model and Its Application to Vehicle Testing

Vehicle-Related Scene Understanding Using Deep Learning

An Improved Deep Network-Based Scene Classification Method for Self-Driving Cars

A Scene Understanding Network Based on Driving Scene

Advancing Traffic Simulation Precision and Scalability: A Data-Driven Approach Utilizing Deep Neural Networks

Automatic Virtual Test Technology for Intelligent Driving Systems Considering Both Coverage and Efficiency

A Digital Twin-based Testing and Data Collection System for Autonomous Driving in Extreme Traffic Scenarios

Automated Scenario Generation and Iterative Regression Testing Method for Autonomous Driving Systems

Multi-Modal Sensor Fusion-Based Deep Neural Network for End-to-End Autonomous Driving With Scene Understanding

From the Lab to the Street: Solving the Challenge of Accelerating Automated Vehicle Testing

Data-driven multi-dimension driving safety evaluation for real-world electric vehicles

Network based on real traffic experimental data

Research on Visual Perception Technology of Autonomous Driving Based on Improved Convolutional Neural Network

Modeling takeover time for non-driving related tasks using hybrid deep neural networks in conditionally automated vehicles

Real-Time Evaluation of Perception Uncertainty and Validity Verification of Autonomous Driving