Abstract:Automated driving is essential for developing and deploying intelligent transportation systems. However, unavoidable sensor noises or perception errors may cause an automated vehicle to adopt suboptimal driving policies or even lead to catastrophic failures. Additionally, the automated driving longitudinal and lateral decision-making behaviors (e.g., driving speed and lane changing decisions) are coupled, that is, when one of them is perturbed by unknown external disturbances, it causes changes or even performance degradation in the other. The presence of both challenges significantly curtails the potential of automated driving. Here, to coordinate the longitudinal and lateral driving decisions of an automated vehicle while ensuring policy robustness against observational uncertainties, we propose a novel robust coordinated decision-making technique via robust multiagent reinforcement learning. Specifically, the automated driving longitudinal and lateral decisions under observational perturbations are modeled as a constrained robust multiagent Markov decision process. Meanwhile, a nonlinear constraint setting with Kullback–Leibler divergence is developed to keep the variation of the driving policy perturbed by stochastic perturbations within bounds. Additionally, a robust multiagent policy optimization approach is proposed to approximate the optimal robust coordinated driving policy. Finally, we evaluate the proposed robust coordinated decision-making method in three highway scenarios with different traffic densities. Quantitatively, in the absence of noises, the proposed method achieves an approximate average enhancement of 25.58% in traffic efficiency and 91.31% in safety compared to all baselines across the three scenarios. In the presence of noises, our technique improves traffic efficiency and safety by an approximate average of 30.81% and 81.02% compared to all baselines in the three scenarios, respectively. The results demonstrate that the proposed approach is capable of improving automated driving performance and ensuring policy robustness against observational uncertainties.

VN-MADDPG: A Variable-Noise-Based Multi-Agent Reinforcement Learning Algorithm for Autonomous Vehicles at Unsignalized Intersections

Decision-Making Models for Autonomous Vehicles at Unsignalized Intersections Based on Deep Reinforcement Learning.

Cooperative Decision-Making for CAVs at Unsignalized Intersections: A MARL Approach with Attention and Hierarchical Game Priors

Cooperative Multiagent Deep Deterministic Policy Gradient (comaddpg) for Intelligent Connected Transportation with Unsignalized Intersection

Decision-Making for Autonomous Vehicles in Random Task Scenarios at Unsignalized Intersection Using Deep Reinforcement Learning

Decision-Making Models Based on Meta-Reinforcement Learning for Intelligent Vehicles at Urban Intersections

Robust Multiagent Reinforcement Learning Toward Coordinated Decision-Making of Automated Vehicles

Autonomous Vehicle Decision-Making at Unsignalized Intersection via Deep Reinforcement Learning

Autonomous Vehicles Roundup Strategy by Reinforcement Learning with Prediction Trajectory

A Decision-Making Model for Autonomous Vehicles at Urban Intersections Based on Conflict Resolution

AF-DQN: A Large-Scale Decision-Making Method at Unsignalized Intersections with Safe Action Filter and Efficient Exploratory Training Strategy

Deep Reinforcement Learning Enabled Decision-Making for Autonomous Driving at Intersections

Modeling Interactions of Autonomous/Manual Vehicles and Pedestrians with a Multi-Agent Deep Deterministic Policy Gradient

Autonomous Vehicle Decision-Making Framework for Considering Malicious Behavior at Unsignalized Intersections

Towards Safe Decision-Making for Autonomous Vehicles at Unsignalized Intersections

A Rear Anti-Collision Decision-Making Methodology Based on Deep Reinforcement Learning for Autonomous Commercial Vehicles

Deep Reinforcement Learning Based Game-Theoretic Decision-Making for Autonomous Vehicles

Proximal Policy Optimization Through a Deep Reinforcement Learning Framework for Multiple Autonomous Vehicles at a Non-Signalized Intersection

High-Speed Ramp Merging Behavior Decision for Autonomous Vehicles Based on Multi-Agent Reinforcement Learning

Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions

Autonomous Vehicles' Decision-Making Behavior in Complex Driving Environments Using Deep Reinforcement Learning