Abstract:We address the problem of coordination and control of Connected and Automated Vehicles (CAVs) in the presence of imperfect observations in mixed traffic environment. A commonly used approach is learning-based decision-making, such as reinforcement learning (RL). However, most existing safe RL methods suffer from two limitations: (i) they assume accurate state information, and (ii) safety is generally defined over the expectation of the trajectories. It remains challenging to design optimal coordination between multi-agents while ensuring hard safety constraints under system state uncertainties (e.g., those that arise from noisy sensor measurements, communication, or state estimation methods) at every time step. We propose a safety guaranteed hierarchical coordination and control scheme called Safe-RMM to address the challenge. Specifically, the high-level coordination policy of CAVs in mixed traffic environment is trained by the Robust Multi-Agent Proximal Policy Optimization (RMAPPO) method. Though trained without uncertainty, our method leverages a worst-case Q network to ensure the model's robust performances when state uncertainties are present during testing. The low-level controller is implemented using model predictive control (MPC) with robust Control Barrier Functions (CBFs) to guarantee safety through their forward invariance property. We compare our method with baselines in different road networks in the CARLA simulator. Results show that our method provides best evaluated safety and efficiency in challenging mixed traffic environments with uncertainties.

Adaptive control for multi-agent systems with actuator fault via reinforcement learning and its application on multi-unmanned surface vehicle

Multi-Agent Reinforcement Learning Control for Consensus Problems of Uncertain Nonlinear Multi-Agent Systems

Adaptive Reinforcement Learning for Fault-Tolerant Optimal Consensus Control of Nonlinear Canonical Multiagent Systems With Actuator Loss of Effectiveness

Adaptive resilient containment control using reinforcement learning for nonlinear stochastic multi-agent systems under sensor faults

Event-based adaptive fixed-time optimal control for saturated fault-tolerant nonlinear multiagent systems via reinforcement learning algorithm

Adaptive Optimal Consensus Control of Multiagent Systems with Unknown Dynamics and Disturbances Via Reinforcement Learning

ADP-based fault-tolerant consensus control for multiagent systems with irregular state constraints

Robust Adaptive Fault-Tolerant Control for Path Maneuvering of Autonomous Surface Vehicles with Actuator Faults Based on the Noncooperative Game Strategy

Fault-tolerant Control for Unmanned Aerial Vehicle Using Deep Reinforcement Learning

Safety Guaranteed Robust Multi-Agent Reinforcement Learning with Hierarchical Control for Connected and Automated Vehicles

Performance-Guaranteed Adaptive Optimized Control of Intelligent Surface Vehicle Using Reinforcement Learning

Distributed fault‐tolerant time‐varying formation control of heterogeneous multi‐agent systems

Distributed Fault-Tolerant Containment Control Protocols for the Discrete-Time Multiagent Systems via Reinforcement Learning Method

Robust Adaptive Fault-Tolerant Control for Unmanned Surface Vehicle Via the Multiplied Event-Triggered Mechanism

Adaptive Optimal Tracking Control for Uncertain Unmanned Surface Vessel Via Reinforcement Learning

Finite-time consensus control for multi-agent systems with full-state constraints and actuator failures

Adaptive fault identification and reconfigurable fault‐tolerant control for unmanned surface vehicle with actuator magnitude and rate faults

Learning‐based hierarchical control for fault‐tolerant cooperation of networked marine surface vehicles

Neural-Network-Based Adaptive Fault-Tolerant Cooperative Control of Heterogeneous Multiagent Systems With Multiple Faults and DoS Attacks

Neural Network-Based Adaptive Consensus Control for a Class of Nonaffine Nonlinear Multiagent Systems with Actuator Faults.

Robust Cooperative Fault-Tolerant Control for Discrete-Time Multi-Agent Systems With Uncertainties and Actuator Faults