Abstract:In recent years, end-to-end imitation learning-based deep neural networks (DNN) have been successfully applied to several autonomous driving tasks. Meanwhile, however, the large uncertainties with respect to the reliability of the DNN’s decision-making mechanism significantly limit the application of DNN-based approaches in the safety-critical domain of autonomous driving. In this work, a comprehensive analytical method of the DNN’s decision-making mechanism is proposed for developing a reliable imitation learning-based end-to-end visual solution for automated lane keeping on the highway in a co-simulation environment. During the mechanism analysis, three progressive and crucial questions covering from the DNN’s decision-making basis to decision itself can be explicitly answered using the modified algorithms of Explainable artificial intelligence (XAI) and the theoretical knowledge of vehicle dynamics; During the model development, the DNN’s architecture is stepwise improved based on the intermediate analytical and test results in the imperfect and failure cases, aimed at increasing the DNN’s lane keeping imitation accuracy and robustness from a superficial point of view as well as enhancing the reliability of the DNN’s decision-making mechanism from an essential point of view. The DNN’s lane keeping performance is quantitatively evaluated based on a proposed evaluation method considering 3 vital perspectives regarding vehicle dynamics. The ultimate test results show that the final proposed DNN model, which has a reliable decision-making mechanism, achieves a satisfactory lane keeping performance with a high imitation accuracy and robustness, and outperforms the state-of-the-art lane keeping approaches based on the end-to-end DNN models and the traditional modular solution.

Imitation Learning for Vision-based Lane Keeping Assistance

Autonomous Driving with Human Guided Image Feature Extraction

A Deep Learning Method for Lane Changing Situation Assessment and Decision Making.

Vision-based reinforcement learning for lane-tracking control

End-to-end Driving via Conditional Imitation Learning

Reliability of Deep Neural Networks for an End-to-End Imitation Learning-Based Lane Keeping

Safe Imitation Learning on Real-Life Highway Data for Human-like Autonomous Driving

A Sim-to-Real Vision-based Lane Keeping System for a 1:10-scale Autonomous Vehicle

Vision-Based Lane-Changing Behavior Detection Using Deep Residual Neural Network

Scaling Vision-based End-to-End Driving with Multi-View Attention Learning

Conditional Affordance Learning for Driving in Urban Environments

A Deep Reinforcement Learning Strategy for Surrounding Vehicles-Based Lane-Keeping Control

Driving Behavior Analysis of Intelligent Vehicle System for Lane Detection Using Vision-Sensor

End-to-End Deep Reinforcement Learning for Lane Keeping Assist

A method based on Vision Transformer and multiple image information for vehicle lane-changing recognition in mixed traffic and connected environment

End-to-End Learning for High Precision Lane Keeping via Multi-State Model

Deep Multi-Sensor Lane Detection

Learning On-Road Visual Control for Self-Driving Vehicles with Auxiliary Tasks

A Vision-Based Lane Detection Approach for Autonomous Vehicles Using a Convolutional Neural Network Architecture

Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safety

Imitation Learning for End to End Vehicle Longitudinal Control with Forward Camera