Abstract:In recent years, autonomous driving algorithms using low-cost vehicle-mounted cameras have attracted increasing endeavors from both academia and industry. There are multiple fronts to these endeavors, including object detection on roads, 3-D reconstruction etc., but in this work we focus on a vision-based model that directly maps raw input images to steering angles using deep networks. This represents a nascent research topic in computer vision. The technical contributions of this work are three-fold. First, the model is learned and evaluated on real human driving videos that are time-synchronized with other vehicle sensors. This differs from many prior models trained from synthetic data in racing games. Second, state-of-the-art models, such as PilotNet, mostly predict the wheel angles independently on each video frame, which contradicts common understanding of driving as a stateful process. Instead, our proposed model strikes a combination of spatial and temporal cues, jointly investigating instantaneous monocular camera observations and vehicle's historical states. This is in practice accomplished by inserting carefully-designed recurrent units (e.g., LSTM and Conv-LSTM) at proper network layers. Third, to facilitate the interpretability of the learned model, we utilize a visual back-propagation scheme for discovering and visualizing image regions crucially influencing the final steering prediction. Our experimental study is based on about 6 hours of human driving data provided by Udacity. Comprehensive quantitative evaluations demonstrate the effectiveness and robustness of our model, even under scenarios like drastic lighting changes and abrupt turning. The comparison with other state-of-the-art models clearly reveals its superior performance in predicting the due wheel angle for a self-driving car.

Multi-task Driver Steering Behaviour Modeling Using Time-Series Transformer

Driver Steering Behaviour Modelling Based on Neuromuscular Dynamics and Multi-Task Time-Series Transformer

Continuous Driver Steering Intention Prediction Considering Neuromuscular Dynamics and Driving Postures

Hybrid-Learning-Based Driver Steering Intention Prediction Using Neuromuscular Dynamics.

Beyond Traditional Driving Scenes: A Robotic-Centric Paradigm for 2D+3D Human Tracking Using Siamese Transformer Network

Pattern Recognition and Characterization of Upper Limb Neuromuscular Dynamics during Driver-Vehicle Interactions

Drivers' Manoeuvre Modelling and Prediction for Safe HRI

Data-driven Steering Torque Behaviour Modelling with Hidden Markov Models

End-To-End Driving Model For Steering Control Of Autonomous Vehicles With Future Spatiotemporal Features

A Unified Multi-scale and Multi-task Learning Framework for Driver Behaviors Reasoning

Personalized Driver Model and Its Application to Vehicle Testing

Model Predictive Control of Steering Torque in Shared Driving of Autonomous Vehicles.

Modeling Lateral Control Behavior of Driver for Manual Takeover of Automated Vehicles During Pedestrian Collision Avoidance

End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perception

Human-vehicle dynamic model with driver’s neuromuscular characteristic for shared control of autonomous vehicle

Modeling and analysis of driver behavior under shared control through weighted visual and haptic guidance

Deep Steering: Learning End-to-End Driving Model from Spatial and Temporal Visual Cues

Human-Like Decision Making of Artificial Drivers in Intelligent Transportation Systems: An End-to-End Driving Behavior Prediction Approach

Driving Maneuvers Prediction Based Autonomous Driving Control by Deep Monte Carlo Tree Search

Human Insights Driven Latent Space for Different Driving Perspectives: A Unified Encoder for Efficient Multi-Task Inference

Multimodal Manoeuvre and Trajectory Prediction for Automated Driving on Highways Using Transformer Networks