Abstract:Traffic Signal Control (TSC) plays a crucial role in mitigating congestion. The extensive integration of Deep Reinforcement Learning (DRL) into TSC, fueled by the advancements in artificial intelligence, has yielded remarkable efficacy. Nevertheless, prevailing DRL models are typically developed within homogeneous road networks, characterized by identical intersection types and relatively balanced traffic flow patterns. This predominant approach often results in an oversight of the impact of intersection heterogeneity, encompassing variations in intersection geometric structure, traffic demands, signal timing design, and other pertinent factors, on action selections by researchers. This paper addresses the aforementioned gap by introducing a value-decomposition-based spatiotemporal graph attention multi-agent DRL model (MARL_SGAT) that expressly accommodates intersection heterogeneity. Specifically, a heterogeneous correlation index is designed using road network structural parameters and serves to formulate a heterogeneous reward function, enabling the quantification of action execution improvements in diverse road network environments. To address the intricate spatiotemporal features of traffic flows within the heterogeneous road network, the model strategically selects and incorporates these features during the formulation by leveraging a spatiotemporal graph attention network. On these bases, double dual networks are introduced to convert the individual-global-max constraint of action selection into a value range constraint of the action advantage function, thereby facilitating model learning. Simulation results showcase the superior performance of the MARL_SGAT algorithm when compared to seven baseline algorithms. Specifically, the algorithm manifests in reduced average vehicle delays, decreased number of stops, and elevated travel speeds within the controlled road network. These advantages are particularly conspicuous in heterogeneous road network environments.

Autonomous Intersection Management with Heterogeneous Vehicles: A Multi-Agent Reinforcement Learning Approach

TraCo: Learning Virtual Traffic Coordinator for Cooperation with Multi-Agent Reinforcement Learning.

Multi-Agent Constrained Policy Optimization for Conflict-Free Management of Connected Autonomous Vehicles at Unsignalized Intersections

A Distributed Approach to Autonomous Intersection Management via Multi-Agent Reinforcement Learning

D-HAL: Distributed Hierarchical Adversarial Learning for Multi-Agent Interaction in Autonomous Intersection Management

HARL: A Novel Hierachical Adversary Reinforcement Learning for Automoumous Intersection Management

A Multi-Agent Reinforcement Learning Approach For Safe and Efficient Behavior Planning Of Connected Autonomous Vehicles

iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning

Multi-agent Deep Reinforcement Learning collaborative Traffic Signal Control method considering intersection heterogeneity

Vehicular intelligent collaborative intersection driving decision algorithm in Internet of Vehicles

Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic

Multi-agent reinforcement learning for cooperative lane changing of connected and autonomous vehicles in mixed traffic

Coordination for Connected and Automated Vehicles at Non-Signalized Intersections: A Value Decomposition-Based Multiagent Deep Reinforcement Learning Approach

Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospects

Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion

MARP: A Cooperative Multiagent DRL System for Connected Autonomous Vehicle Platooning

Multi-Agent Deep Reinforcement Learning for Urban Traffic Light Control in Vehicular Networks

Heuristic-Based Multi-Agent Deep Reinforcement Learning Approach for Coordinating Connected and Automated Vehicles at Non-Signalized Intersection

Combat Urban Congestion via Collaboration: Heterogeneous GNN-based MARL for Coordinated Platooning and Traffic Signal Control

Multi-Vehicle Cooperative Decision-Making in Merging Area Based on Deep Multi-Agent Reinforcement Learning

Communication-Efficient Decentralized Multi-Agent Reinforcement Learning for Cooperative Adaptive Cruise Control