Abstract:Automated driving is essential for developing and deploying intelligent transportation systems. However, unavoidable sensor noises or perception errors may cause an automated vehicle to adopt suboptimal driving policies or even lead to catastrophic failures. Additionally, the automated driving longitudinal and lateral decision-making behaviors (e.g., driving speed and lane changing decisions) are coupled, that is, when one of them is perturbed by unknown external disturbances, it causes changes or even performance degradation in the other. The presence of both challenges significantly curtails the potential of automated driving. Here, to coordinate the longitudinal and lateral driving decisions of an automated vehicle while ensuring policy robustness against observational uncertainties, we propose a novel robust coordinated decision-making technique via robust multiagent reinforcement learning. Specifically, the automated driving longitudinal and lateral decisions under observational perturbations are modeled as a constrained robust multiagent Markov decision process. Meanwhile, a nonlinear constraint setting with Kullback–Leibler divergence is developed to keep the variation of the driving policy perturbed by stochastic perturbations within bounds. Additionally, a robust multiagent policy optimization approach is proposed to approximate the optimal robust coordinated driving policy. Finally, we evaluate the proposed robust coordinated decision-making method in three highway scenarios with different traffic densities. Quantitatively, in the absence of noises, the proposed method achieves an approximate average enhancement of 25.58% in traffic efficiency and 91.31% in safety compared to all baselines across the three scenarios. In the presence of noises, our technique improves traffic efficiency and safety by an approximate average of 30.81% and 81.02% compared to all baselines in the three scenarios, respectively. The results demonstrate that the proposed approach is capable of improving automated driving performance and ensuring policy robustness against observational uncertainties.

OPTIMA: Optimized Policy for Intelligent Multi-Agent Systems Enables Coordination-Aware Autonomous Vehicles

Multi-Agent Reinforcement Learning-Based Autonomous Intersection Management Protocol with Attention Mechanism

Autonomous Intersection Management with Heterogeneous Vehicles: A Multi-Agent Reinforcement Learning Approach

Integrated Operations Strategies for Shared and Privately-Owned Autonomous Vehicles: A Deep Reinforcement Learning Framework

Multi-Agent Reinforcement Learning for Unmanned Aerial Vehicle Coordination by Multi-Critic Policy Gradient Optimization

Robust Multiagent Reinforcement Learning Toward Coordinated Decision-Making of Automated Vehicles

Efficient Domain Coverage for Vehicles with Second-Order Dynamics via Multi-Agent Reinforcement Learning

Multi-Agent Constrained Policy Optimization for Conflict-Free Management of Connected Autonomous Vehicles at Unsignalized Intersections

Towards Cooperative Driving among Heterogeneous CAVs: A Safe Multi-Agent Reinforcement Learning Approach

A Fast Optimal Coordination Method for Multiagent in Complex Environment

Coordination for Connected and Automated Vehicles at Non-Signalized Intersections: A Value Decomposition-Based Multiagent Deep Reinforcement Learning Approach

Fast Multi-Class Vehicle Cooperative Path Optimization in Complex Urban V2X Transportation: A Novel Parallel Multi-Agent Reinforcement Learning Approach

Towards Socially Responsive Autonomous Vehicles: A Reinforcement Learning Framework with Driving Priors and Coordination Awareness

An Optimization-Based Cooperative Path-Following Framework for Multiple Robotic Vehicles

Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Critic

Robustness and Adaptability of Reinforcement Learning-Based Cooperative Autonomous Driving in Mixed-Autonomy Traffic

A Multiobjective Cooperative Driving Framework Based on Evolutionary Algorithm and Multitask Learning

Safety Guaranteed Robust Multi-Agent Reinforcement Learning with Hierarchical Control for Connected and Automated Vehicles

Multi-agent reinforcement learning for ecological car-following control in mixed traffic

A Multi-Agent Deep Reinforcement Learning Coordination Framework for Connected and Automated Vehicles at Merging Roadways

A Multi-agent Reinforcement Learning Based Control Method for Connected and Autonomous Vehicles in A Mixed Platoon