Abstract:A safe and efficient decision-making system is crucial for autonomous vehicles. However, the complexity of driving environments limits the effectiveness of many rule-based and machine learning approaches. Reinforcement Learning, with its robust self-learning capabilities and environmental adaptability, offers a promising solution to these challenges. Nevertheless, safety and efficiency concerns during training hinder its widespread application. To address these concerns, we propose a novel RL framework, Simple to Complex Collaborative Decision (S2CD). First, we rapidly train the teacher model in a lightweight simulation environment. In the more complex and realistic environment, the teacher intervenes when the student agent exhibits suboptimal behavior by assessing actions' value to avert dangers. We also introduce an RL algorithm called Adaptive Clipping Proximal Policy Optimization (ACPPO), which combines samples from both teacher and student policies and employs dynamic clipping strategies based on sample importance. This approach improves sample efficiency while effectively alleviating data imbalance. Additionally, we employ the Kullback-Leibler divergence as a policy constraint, transforming it into an unconstrained problem with the Lagrangian method to accelerate the student's learning. Finally, a gradual weaning strategy ensures that the student learns to explore independently over time, overcoming the teacher's limitations and maximizing performance. Simulation experiments in highway lane-change scenarios show that the S2CD framework enhances learning efficiency, reduces training costs, and significantly improves safety compared to state-of-the-art algorithms. This framework also ensures effective knowledge transfer between teacher and student models, even with a suboptimal teacher, the student achieves superior performance, demonstrating the robustness and effectiveness of S2CD.

Uncertainty Sequence Modeling Approach for Safe and Effective Autonomous Driving

Uncertainty-Aware Model-Based Reinforcement Learning: Methodology and Application in Autonomous Driving

Uncertainty-Aware Model-Based Reinforcement Learning with Application to Autonomous Driving

Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving

Knowledge Transfer from Simple to Complex: A Safe and Efficient Reinforcement Learning Framework for Autonomous Driving Decision-Making

Uncertainty-Aware Model-Based Offline Reinforcement Learning for Automated Driving

Safe Decision Controller for Autonomous DrivingBased on Deep Reinforcement Learning inNondeterministic Environment

Tactical Decision-Making in Autonomous Driving by Reinforcement Learning with Uncertainty Estimation

Lyapunov-based uncertainty-aware safe reinforcement learning

Safe-State Enhancement Method for Autonomous Driving Via Direct Hierarchical Reinforcement Learning.

Autonomous Motion Decision-making based on Deep Reinforcement Learning for Autonomous Driving

Uncertainty-Aware Decision-Making for Autonomous Driving at Uncontrolled Intersections

Towards Robust Decision-Making for Autonomous Driving on Highway

Enhancing System-Level Safety in Mixed-Autonomy Platoon via Safe Reinforcement Learning

Autonomous Highway Driving using Deep Reinforcement Learning

Deep Reinforcement Learning Based Decision-Making Strategy of Autonomous Vehicle in Highway Uncertain Driving Environments

A Safe and Efficient Lane Change Decision-Making Strategy of Autonomous Driving Based on Deep Reinforcement Learning

Reinforcement Learning Based Safe Decision Making for Highway Autonomous Driving

Safety Guaranteed Robust Multi-Agent Reinforcement Learning with Hierarchical Control for Connected and Automated Vehicles

A Safe and Efficient Self-evolving Algorithm for Decision-making and Control of Autonomous Driving Systems