Abstract:Self-driving technology is advancing rapidly --- albeit with significant challenges and limitations. This progress is largely due to recent developments in deep learning algorithms. To date, however, there has been no systematic comparison of how different deep learning architectures perform at such tasks, or an attempt to determine a correlation between classification performance and performance in an actual vehicle, a potentially critical factor in developing self-driving systems. Here, we introduce the first controlled comparison of multiple deep-learning architectures in an end-to-end autonomous driving task across multiple testing conditions. We compared performance, under identical driving conditions, across seven architectures including a fully-connected network, a simple 2 layer CNN, AlexNet, VGG-16, Inception-V3, ResNet, and an LSTM by assessing the number of laps each model was able to successfully complete without crashing while traversing an indoor racetrack. We compared performance across models when the conditions exactly matched those in training as well as when the local environment and track were configured differently and objects that were not included in the training dataset were placed on the track in various positions. In addition, we considered performance using several different data types for training and testing including single grayscale and color frames, and multiple grayscale frames stacked together in sequence. With the exception of a fully-connected network, all models performed reasonably well (around or above 80\%) and most very well (~95\%) on at least one input type but with considerable variation across models and inputs. Overall, AlexNet, operating on single color frames as input, achieved the best level of performance (100\% success rate in phase one and 55\% in phase two) while VGG-16 performed well most consistently across image types.

Systematic Testing of Convolutional Neural Networks for Autonomous Driving

Applying adversarial networks to increase the data efficiency and reliability of Self-Driving Cars

DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars

A Convolutional Neural Network Approach Towards Self-Driving Cars

NeurAll: Towards a Unified Visual Perception Model for Automated Driving

Reliability Evaluation of Visualization Performance of Convolutional Neural Network Models for Automated Driving

Research on Visual Perception Technology of Autonomous Driving Based on Improved Convolutional Neural Network

A Systematic Comparison of Deep Learning Architectures in an Autonomous Vehicle

Driving Behaviour Characterisation by Using Phase‐space Reconstruction and Pre‐trained Convolutional Neural Network

Coverage Testing of Deep Learning Models using Dataset Characterization

Convolution Neural Network Application for Road Asset Detection and Classification in LiDAR Point Cloud

Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components

An Analysis of Adversarial Attacks and Defenses on Autonomous Driving Models

Vehicle detection systems for intelligent driving using deep convolutional neural networks

When, Where and How Does It Fail? A Spatial-Temporal Visual Analytics Approach for Interpretable Object Detection in Autonomous Driving.

Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving Cars

An Empirical Evaluation of Deep Learning on Highway Driving

Cross-modal Matching CNN for Autonomous Driving Sensor Data Monitoring

Impact of White-Box Adversarial Attacks on Convolutional Neural Networks

Closed-loop Analysis of Vision-based Autonomous Systems: A Case Study

Vision-Based Obstacle Detection and Collision Prevention in Self-Driving Cars