Abstract:Ensuring safety and achieving human-level driving performance remain challenges for autonomous vehicles, especially in safety-critical situations. As a key component of artificial intelligence, reinforcement learning is promising and has shown great potential in many complex tasks; however, its lack of safety guarantees limits its real-world applicability. Hence, further advancing reinforcement learning, especially from the safety perspective, is of great importance for autonomous driving. As revealed by cognitive neuroscientists, the amygdala of the brain can elicit defensive responses against threats or hazards, which is crucial for survival in and adaptation to risky environments. Drawing inspiration from this scientific discovery, we present a fear-neuro-inspired reinforcement learning framework to realize safe autonomous driving through modeling the amygdala functionality. This new technique facilitates an agent to learn defensive behaviors and achieve safe decision making with fewer safety violations. Through experimental tests, we show that the proposed approach enables the autonomous driving agent to attain state-of-the-art performance compared to the baseline agents and perform comparably to 30 certified human drivers, across various safety-critical scenarios. The results demonstrate the feasibility and effectiveness of our framework while also shedding light on the crucial role of simulating the amygdala function in the application of reinforcement learning to safety-critical autonomous driving domains.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to ensure the safety performance of vehicles in autonomous driving, especially when facing high - risk scenarios. Although Reinforcement Learning (RL) performs well in many complex tasks, its lack of safety guarantees limits its practical applications. The paper proposes a Fear - Neuro - Inspired Reinforcement Learning (FNI - RL) framework inspired by the function of the amygdala in the brain, aiming to achieve safe autonomous driving by simulating the function of the amygdala in the brain. This framework enables agents to learn defensive behaviors, reduce violations of safety regulations, and reach a performance level comparable to that of certified human drivers in various high - risk scenarios. Specifically, the main contributions of the paper include: 1. **Introduction of the FNI - RL framework**: Inspired by the fear neurons in the brain, a computational framework is proposed to enhance the safety of autonomous vehicles. 2. **Adversarial imagination technique**: An adversarial imagination technique has been developed. By means of the learned opponent and world models, high - risk situations are simulated, which helps agents deal with unseen risk scenarios and improves the robustness of strategies to uncertainty and non - stationarity. 3. **Fear - Constrained Actor - Critic algorithm (FC - AC)**: A Fear - Constrained Actor - Critic algorithm has been developed, enabling agents to learn defensive driving behaviors, ensuring the safety of decisions, and minimizing violations of safety regulations at the same time. The paper demonstrates the effectiveness of the proposed FNI - RL method through experimental tests. Compared with the existing state - of - the - art AI agents and 30 certified human drivers, this method performs excellently in various high - risk scenarios and significantly improves safety.

Fear-Neuro-Inspired Reinforcement Learning for Safe Autonomous Driving

Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings

Look Before You Leap: Safe Model-Based Reinforcement Learning with Human Intervention

Safe Decision Controller for Autonomous DrivingBased on Deep Reinforcement Learning inNondeterministic Environment

Autonomous Highway Driving using Deep Reinforcement Learning

Towards Brain-inspired System: Deep Recurrent Reinforcement Learning for Simulated Self-driving Agent

Safe Autonomous Driving with Latent Dynamics and State-Wise Constraints

Deep Reinforcement Learning with Enhanced Safety for Autonomous Highway Driving

Safe-State Enhancement Method for Autonomous Driving Via Direct Hierarchical Reinforcement Learning.

Weakly Supervised Reinforcement Learning for Autonomous Highway Driving via Virtual Safety Cages

Enhancing System-Level Safety in Mixed-Autonomy Platoon via Safe Reinforcement Learning

A Safe and Efficient Self-evolving Algorithm for Decision-making and Control of Autonomous Driving Systems

CRASH: Challenging Reinforcement-Learning Based Adversarial Scenarios For Safety Hardening

Self-Awareness Safety of Deep Reinforcement Learning in Road Traffic Junction Driving

Combining Deep Reinforcement Learning and Safety Based Control for Autonomous Driving

A Reinforcement Learning Approach for Enacting Cautious Behaviours in Autonomous Driving System: Safe Speed Choice in the Interaction With Distracted Pedestrians

Neurosymbolic Meta-Reinforcement Lookahead Learning Achieves Safe Self-Driving in Non-Stationary Environments

Decision Making of Autonomous Vehicles in Lane Change Scenarios: Deep Reinforcement Learning Approaches with Risk Awareness

End-to-End Urban Autonomous Driving With Safety Constraints

Dense reinforcement learning for safety validation of autonomous vehicles

Toward a Brain-Inspired System: Deep Recurrent Reinforcement Learning for a Simulated Self-Driving Agent