Abstract:In recent years, autonomous driving algorithms using low-cost vehicle-mounted cameras have attracted increasing endeavors from both academia and industry. There are multiple fronts to these endeavors, including object detection on roads, 3-D reconstruction etc., but in this work we focus on a vision-based model that directly maps raw input images to steering angles using deep networks. This represents a nascent research topic in computer vision. The technical contributions of this work are three-fold. First, the model is learned and evaluated on real human driving videos that are time-synchronized with other vehicle sensors. This differs from many prior models trained from synthetic data in racing games. Second, state-of-the-art models, such as PilotNet, mostly predict the wheel angles independently on each video frame, which contradicts common understanding of driving as a stateful process. Instead, our proposed model strikes a combination of spatial and temporal cues, jointly investigating instantaneous monocular camera observations and vehicle's historical states. This is in practice accomplished by inserting carefully-designed recurrent units (e.g., LSTM and Conv-LSTM) at proper network layers. Third, to facilitate the interpretability of the learned model, we utilize a visual back-propagation scheme for discovering and visualizing image regions crucially influencing the final steering prediction. Our experimental study is based on about 6 hours of human driving data provided by Udacity. Comprehensive quantitative evaluations demonstrate the effectiveness and robustness of our model, even under scenarios like drastic lighting changes and abrupt turning. The comparison with other state-of-the-art models clearly reveals its superior performance in predicting the due wheel angle for a self-driving car.

Generalized two-point visual control model of human steering for accurate state estimation

Regression Model of Autonomous Lateral Rectification with Driver Perception

Analysis of human steering behavior differences in human-in-control and autonomy-in-control driving

Two Nash-equilibrium-based steering control models for representing a driver’s interaction with vehicle automated steering

Learning End-to-End Autonomous Steering Model from Spatial and Temporal Visual Cues

Driver Steering Behavior Model Based on Lane-Keeping Characteristics Analysis.

Vestibular-derived internal models in active self-motion estimation

Automated Vehicle Sideslip Angle Estimation Considering Signal Measurement Characteristic

Deep Steering: Learning End-to-End Driving Model from Spatial and Temporal Visual Cues

Driver-Automation Shared Control: Modeling Driver Behavior by Taking Account of Reliance on Haptic Guidance Steering

Autonomous Cars: Vision based Steering Wheel Angle Estimation

PTV Longitudinal-Lateral State Estimation Considering Unknown Control Inputs and Uncertain Model Parameters

Indirect Shared Control for Cooperative Driving Between Driver and Automation in Steer-by-Wire Vehicles

Modeling and analysis of driver behavior under shared control through weighted visual and haptic guidance

Modeling and Analysis of Driver Behaviour under Shared Control Through Weighted Visual and Haptic Guidance

Model Based on States Preview for Lane Keeping System

Modeling Lateral Control Behaviors of Distracted Drivers for Haptic-Shared Steering System

Learning to Represent and Generalize Lateral Control Deviation Based on Driving Data

Data-driven Steering Torque Behaviour Modelling with Hidden Markov Models

Algebraic Driver Steering Model Parameter Identification

A Control Model For Human Steering Actions During Lane Change Based On Ngsim Dataset