Abstract:Deep reinforcement learning (DRL) is a promising way to achieve human-like autonomous driving. However, the low sample efficiency and difficulty of designing reward functions for DRL would hinder its applications in practice. In light of this, this paper proposes a novel framework to incorporate human prior knowledge in DRL, in order to improve the sample efficiency and save the effort of designing sophisticated reward functions. Our framework consists of three ingredients, namely expert demonstration, policy derivation, and reinforcement learning. In the expert demonstration step, a human expert demonstrates their execution of the task, and their behaviors are stored as state-action pairs. In the policy derivation step, the imitative expert policy is derived using behavioral cloning and uncertainty estimation relying on the demonstration data. In the reinforcement learning step, the imitative expert policy is utilized to guide the learning of the DRL agent by regularizing the KL divergence between the DRL agent's policy and the imitative expert policy. To validate the proposed method in autonomous driving applications, two simulated urban driving scenarios (unprotected left turn and roundabout) are designed. The strengths of our proposed method are manifested by the training results as our method can not only achieve the best performance but also significantly improve the sample efficiency in comparison with the baseline algorithms (particularly 60\% improvement compared to soft actor-critic). In testing conditions, the agent trained by our method obtains the highest success rate and shows diverse and human-like driving behaviors as demonstrated by the human expert.

Driver Behavioral Cloning for Route Following in Autonomous Vehicles Using Task Knowledge Distillation

Knowledge Distillation on Driving Intention Generator: Learn Human-like Semantic Reasoning.

Behavior Cloning for Mini Autonomous Car Path Following

A Shared Control Approach for Autonomous Vehicles via Driver Behaviors Learning

Iterative Imitation Policy Improvement for Interactive Autonomous Driving

High-Level Decision Making for Automated Highway Driving via Behavior Cloning

Implicit predictive behavior cloning for autonomous driving decision-making in urban traffic

Robust Behavioral Cloning for Autonomous Vehicles Using End-to-End Imitation Learning

Learning to Drive Like Human Beings: A Method Based on Deep Reinforcement Learning

Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving

Safe Imitation Learning on Real-Life Highway Data for Human-like Autonomous Driving

Developing Path Planning with Behavioral Cloning and Proximal Policy Optimization for Path-Tracking and Static Obstacle Nudging

Behavioral Cloning Models Reality Check for Autonomous Driving

Driving Tasks Transfer in Deep Reinforcement Learning for Decision-making of Autonomous Vehicles

Improving Behavioural Cloning with Positive Unlabeled Learning

Data Augmented Deep Behavioral Cloning for Urban Traffic Control Operations Under a Parallel Learning Framework

Task-Driven Autonomous Driving: Balanced Strategies Integrating Curriculum Reinforcement Learning and Residual Policy

Human-Like Autonomous Driving on Dense Traffic

Mitigating Causal Confusion in Vector-Based Behavior Cloning for Safer Autonomous Planning

Residual Chain Prediction for Autonomous Driving Path Planning

CCIL: Context-conditioned imitation learning for urban driving