Abstract:In recent years, autonomous driving algorithms using low-cost vehicle-mounted cameras have attracted increasing endeavors from both academia and industry. There are multiple fronts to these endeavors, including object detection on roads, 3-D reconstruction etc., but in this work we focus on a vision-based model that directly maps raw input images to steering angles using deep networks. This represents a nascent research topic in computer vision. The technical contributions of this work are three-fold. First, the model is learned and evaluated on real human driving videos that are time-synchronized with other vehicle sensors. This differs from many prior models trained from synthetic data in racing games. Second, state-of-the-art models, such as PilotNet, mostly predict the wheel angles independently on each video frame, which contradicts common understanding of driving as a stateful process. Instead, our proposed model strikes a combination of spatial and temporal cues, jointly investigating instantaneous monocular camera observations and vehicle's historical states. This is in practice accomplished by inserting carefully-designed recurrent units (e.g., LSTM and Conv-LSTM) at proper network layers. Third, to facilitate the interpretability of the learned model, we utilize a visual back-propagation scheme for discovering and visualizing image regions crucially influencing the final steering prediction. Our experimental study is based on about 6 hours of human driving data provided by Udacity. Comprehensive quantitative evaluations demonstrate the effectiveness and robustness of our model, even under scenarios like drastic lighting changes and abrupt turning. The comparison with other state-of-the-art models clearly reveals its superior performance in predicting the due wheel angle for a self-driving car.

Speed and Steering Angle Prediction for Intelligent Vehicles Based on Deep Belief Network

Lateral and Longitudinal Driving Behavior Prediction Based on Improved Deep Belief Network

A Deep Learning Method for Lane Changing Situation Assessment and Decision Making.

Driving Behaviour Style Study with a Hybrid Deep Learning Framework Based on GPS Data

Hybrid-Learning-Based Driver Steering Intention Prediction Using Neuromuscular Dynamics.

Deep Steering: Learning End-to-End Driving Model from Spatial and Temporal Visual Cues

A Big Data Based Deep Learning Approach for Vehicle Speed Prediction

Parallel Learning-Based Steering Control for Autonomous Driving.

Driver Lane-Changing Behavior Prediction Based on Deep Learning

A Real-Time Vehicle Speed Prediction Method Based on a Lightweight Informer Driven by Big Temporal Data

Anticipating Driving Behavior through Deep Learning-Based Policy Prediction

Freeway Traffic Speed Prediction under the Intelligent Driving Environment: A Deep Learning Approach

Driver Behavior Modeling Using Game Engine and Real Vehicle: A Learning-Based Approach

Bayesian Optimization and Deep Learning forsteering wheel angle prediction

Driver Intent-Based Intersection Autonomous Driving Collision Avoidance Reinforcement Learning Algorithm

Learning End-to-End Autonomous Steering Model from Spatial and Temporal Visual Cues

Continuous Driver Steering Intention Prediction Considering Neuromuscular Dynamics and Driving Postures

Ensemble deep learning-based lane-changing behavior prediction of manually driven vehicles in mixed traffic environments

Learning driving behavior for autonomous vehicles using deep learning based methods*

Deep-Neural-Network-Based Modelling of Longitudinal-Lateral Dynamics to Predict the Vehicle States for Autonomous Driving

Vehicle steering control method based on machine learning