Abstract:In recent years, autonomous driving algorithms using low-cost vehicle-mounted cameras have attracted increasing endeavors from both academia and industry. There are multiple fronts to these endeavors, including object detection on roads, 3-D reconstruction etc., but in this work we focus on a vision-based model that directly maps raw input images to steering angles using deep networks. This represents a nascent research topic in computer vision. The technical contributions of this work are three-fold. First, the model is learned and evaluated on real human driving videos that are time-synchronized with other vehicle sensors. This differs from many prior models trained from synthetic data in racing games. Second, state-of-the-art models, such as PilotNet, mostly predict the wheel angles independently on each video frame, which contradicts common understanding of driving as a stateful process. Instead, our proposed model strikes a combination of spatial and temporal cues, jointly investigating instantaneous monocular camera observations and vehicle's historical states. This is in practice accomplished by inserting carefully-designed recurrent units (e.g., LSTM and Conv-LSTM) at proper network layers. Third, to facilitate the interpretability of the learned model, we utilize a visual back-propagation scheme for discovering and visualizing image regions crucially influencing the final steering prediction. Our experimental study is based on about 6 hours of human driving data provided by Udacity. Comprehensive quantitative evaluations demonstrate the effectiveness and robustness of our model, even under scenarios like drastic lighting changes and abrupt turning. The comparison with other state-of-the-art models clearly reveals its superior performance in predicting the due wheel angle for a self-driving car.

End-to-End Learning for High Precision Lane Keeping via Multi-State Model

A Deep Learning Method for Lane Changing Situation Assessment and Decision Making.

Learning an Efficient and Safe Policy for Highway Driving Using Supervised Learning and Reinforcement Learning.

Longitudinal and Lateral Coupling Model Based End-to-End Learning for Lane Keeping of Self-driving Cars.

An End-to-End Lane Detection Model with Attention and Residual Block

Learning End-to-End Autonomous Steering Model from Spatial and Temporal Visual Cues

Reliability of Deep Neural Networks for an End-to-End Imitation Learning-Based Lane Keeping

Attention-Based Deep Driving Model for Autonomous Vehicles with Surround-View Cameras

Deep Steering: Learning End-to-End Driving Model from Spatial and Temporal Visual Cues

An End-to-End Multi-Task Learning Model for Drivable Road Detection via Edge Refinement and Geometric Deformation

DCFS-based Deep Learning Supervisory Control for Modeling Lane Keeping of Expert Drivers

End-to-End Autonomous Driving Through Dueling Double Deep Q-Network

End-to-End Lane Detection: a Key Point Approach

Deep learning and control algorithms of direct perception for autonomous driving

Lane-changing trajectory prediction based on multi-task learning

End-To-End Driving Model For Steering Control Of Autonomous Vehicles With Future Spatiotemporal Features

Modeling Lane Changes Using Parallel Learning

LaneSegNet: Map Learning with Lane Segment Perception for Autonomous Driving

End-to-End Deep Reinforcement Learning for Lane Keeping Assist

Multi-task UNet architecture for end-to-end autonomous driving

End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners