Abstract:How to efficiently train a high-performance autonomous driving agent remains a realistic and challenging issue. Although in the literature, many techniques, especially deep reinforcement learning (DRL) methods, have been developed, they are mainly computationally inefficient and time-consuming. In recent years, pre-training approaches by utilizing the demonstration data are adopted with DRL to improve the training efficiency and performance of the related methods. However, due to the complexity of these pre-training DRL methods and the strict requirements on the format of the demonstration data, it is still difficult for these methods to be efficiently applicable to autonomous driving scenarios. To alleviate these problems, we propose a new algorithm called Pre-training Deep Reinforcement Learning with Improved Advantage and Loss Function (PDRL). Specifically, for scenarios with high temporal sparsity in continuous action spaces, the pre-training requires only one network to be trained with a designed demonstration regression loss function based on an improved normalized advantage function. In this way, the agent can achieve a higher value when selecting actions similar to the demonstration data and simplify the network's output and computation by using exploration methods that are more suitable for inertial systems. Furthermore, a new prioritization formula is presented to improve the algorithm's performance and convergence speed. Finally, the MetaDrive simulation platform is used to test the performance of the proposed approach and comparisons are made with the existing algorithms, including Soft Actor-Critic (SAC), Twin Delayed Deep Deterministic Policy Gradient (TD3), and Advantage Weighting with Early Termination Actor-Critic (AWET(SAC)).

Efficient and Generalized End-to-end Autonomous Driving System with Latent Deep Reinforcement Learning and Demonstrations

Safe and Generalized end-to-end Autonomous Driving System with Reinforcement Learning and Demonstrations

Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving

End-to-End Urban Autonomous Driving With Safety Constraints

Continuous Reinforcement Learning From Human Demonstrations With Integrated Experience Replay For Autonomous Driving

End-to-End Urban Driving by Imitating a Reinforcement Learning Coach

Interpretable End-to-End Urban Autonomous Driving With Latent Deep Reinforcement Learning

A Deep Reinforcement Learning Algorithm with Expert Demonstrations and Supervised Loss and Its Application in Autonomous Driving

Autonomous Driving Control Based on the Technique of Semantic Segmentation

Safe Autonomous Driving with Latent Dynamics and State-Wise Constraints

Carl-Lead: Lidar-based End-to-End Autonomous Driving with Contrastive Deep Reinforcement Learning

Deductive Reinforcement Learning for Visual Autonomous Urban Driving Navigation.

Autonomous Highway Driving using Deep Reinforcement Learning

DQ-GAT: Towards Safe and Efficient Autonomous Driving With Deep Q-Learning and Graph Attention Networks

Autonomous driving policy learning from demonstration using regression loss function

DeepGoal: Learning to drive with driving intention from human control demonstration

Deep Reinforcement Learning for Autonomous Driving Based on Safety Experience Replay

Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving

Deep Reinforcement Learning for Autonomous Driving with an Auxiliary Actor Discriminator

Large Language Model guided Deep Reinforcement Learning for Decision Making in Autonomous Driving