Abstract:In the field of autonomous driving, developing safe and trustworthy autonomous driving policies remains a significant challenge. Recently, Reinforcement Learning with Human Feedback (RLHF) has attracted substantial attention due to its potential to enhance training safety and sampling efficiency. Nevertheless, existing RLHF-enabled methods often falter when faced with imperfect human demonstrations, potentially leading to training oscillations or even worse performance than rule-based approaches. Inspired by the human learning process, we propose Physics-enhanced Reinforcement Learning with Human Feedback (PE-RLHF). This novel framework synergistically integrates human feedback (e.g., human intervention and demonstration) and physics knowledge (e.g., traffic flow model) into the training loop of reinforcement learning. The key advantage of PE-RLHF is its guarantee that the learned policy will perform at least as well as the given physics-based policy, even when human feedback quality deteriorates, thus ensuring trustworthy safety improvements. PE-RLHF introduces a Physics-enhanced Human-AI (PE-HAI) collaborative paradigm for dynamic action selection between human and physics-based actions, employs a reward-free approach with a proxy value function to capture human preferences, and incorporates a minimal intervention mechanism to reduce the cognitive load on human mentors. Extensive experiments across diverse driving scenarios demonstrate that PE-RLHF significantly outperforms traditional methods, achieving state-of-the-art (SOTA) performance in safety, efficiency, and generalizability, even with varying quality of human feedback. The philosophy behind PE-RLHF not only advances autonomous driving technology but can also offer valuable insights for other safety-critical domains. Demo video and code are available at: \<a class="link-external link-https" href="https://zilin-huang.github.io/PE-RLHF-website/" rel="external noopener nofollow">this https URL</a>

Trustworthy safety improvement for autonomous driving using reinforcement learning

Autonomous Highway Driving using Deep Reinforcement Learning

Confidence-Aware Reinforcement Learning for Self-Driving Cars

Towards Robust Decision-Making for Autonomous Driving on Highway

Towards Robust Decision-Making for Autonomous Highway Driving Based on Safe Reinforcement Learning

Trustworthy Human-AI Collaboration: Reinforcement Learning with Human Feedback and Physics Knowledge for Safe Autonomous Driving

Autonomous highway driving using reinforcement learning with safety check system based on time-to-collision

Decision-making Strategy on Highway for Autonomous Vehicles using Deep Reinforcement Learning

Safe-State Enhancement Method for Autonomous Driving Via Direct Hierarchical Reinforcement Learning.

Task-Driven Autonomous Driving: Balanced Strategies Integrating Curriculum Reinforcement Learning and Residual Policy

Reinforcement Learning based Control of Imitative Policies for Near-Accident Driving

Comprehensive Training and Evaluation on Deep Reinforcement Learning for Automated Driving in Various Simulated Driving Maneuvers

Trustworthy autonomous driving via defense-aware robust reinforcement learning against worst-case observational perturbations

From Naturalistic Traffic Data to Learning-Based Driving Policy: A Sim-to-Real Study

A Safe and Efficient Lane Change Decision-Making Strategy of Autonomous Driving Based on Deep Reinforcement Learning

Road Traffic Law Adaptive Decision-making for Self-Driving Vehicles

Deep Reinforcement Learning with Enhanced Safety for Autonomous Highway Driving

Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving

Belief State Separated Reinforcement Learning for Autonomous Vehicle Decision Making under Uncertainty.

Long and Short-Term Constraints Driven Safe Reinforcement Learning for Autonomous Driving

Reinforcement Learning Based Safe Decision Making for Highway Autonomous Driving