Abstract:Reinforcement learning (RL) is a promising way to achieve human-like autonomous driving (HAD) in complex and dynamic traffic, but faces challenges such as low sample efficiency, partial observability, and sim2real transfer. In light of this, a comprehensive solution for RL-driven HAD is established. First, an efficient training scheme called Deep Recurrent Q-learning from demonstration algorithm (DRQfD) is proposed for lane-changing decision-making to address the low sample efficiency in RL and the poor generalization capability in Imitation Learning (IL). The inherent LSTM structure potentially learns to predict future states of surrounding vehicles, helping to address the partially observable problem in autonomous driving (AD). Second, to reduce the sim2real gap, a twin high-fidelity simulator is built based on ROS-Gazebo for simulating LiDAR sensing, model training, and evaluations. Domain randomization is used to improve the robustness and generalization ability, making it easier for the model to be transferred to real-world scenarios. In addition, for the multi-objective optimization and imbalanced data issues in this scenario, a hierarchical decision-making framework is proposed to decompose the complex decision-making problem into several subtasks, making the driving policies easier to converge. To avoid the excessive dependence of the decision-making module on the output of perception module in modular systems, we train each modularized skill in an end-to-end manner. Moreover, we compare our method with a vanilla RL method to show improvement in learning efficiency. Comparisons between RL-based model and IL baseline in terms of safety, travel efficiency, and human-likeness are also given. To further validate the generalization ability of our model, we test the model on real traffic dataset. Finally, we implement the RL model on physical cars to demonstrate the performance of sim2real transfer.

Learning to Simulate Vehicle Trajectories from Demonstrations

Learning to Simulate on Sparse Trajectory Data

Imitation Learning of Hierarchical Driving Model: from Continuous Intention to Continuous Trajectory

Objective-aware Traffic Simulation via Inverse Reinforcement Learning

LASIL: Learner-Aware Supervised Imitation Learning For Long-term Microscopic Traffic Simulation

TrajGen: Generating Realistic and Diverse Trajectories With Reactive and Feasible Agent Behaviors for Autonomous Driving

A Systematic Solution of Human Driving Behavior Modeling and Simulation for Automated Vehicle Studies

Learning Interactive Driving Policies via Data-driven Simulation

Simulating Dynamic Driving Behavior in Simulation Test for Unmanned Vehicles via Multi-Sensor Data.

Utilizing Hybrid Trajectory Prediction Models to Recognize Highly Interactive Traffic Scenarios

Autonomous Driving Simulation for Unmanned Vehicles.

Traffic-Aware Autonomous Driving with Differentiable Traffic Simulation

Predicting and Optimizing City-Scale Road Traffic Dynamics Using Trajectories of Individual Vehicles.

InterSim: Interactive Traffic Simulation Via Explicit Relation Modeling

Modelling two-dimensional driving behaviours at unsignalised intersection using multi-agent imitation learning

Exploring the trade off between human driving imitation and safety for traffic simulation

From Naturalistic Traffic Data to Learning-Based Driving Policy: A Sim-to-Real Study

TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarios.

Learning Realistic Traffic Agents in Closed-loop

Learning two-dimensional merging behaviour from vehicle trajectories with imitation learning

A Learning-based Stochastic Driving Model for Autonomous Vehicle Testing