Abstract:Multi-agent deep reinforcement learning (MDRL) has been widely applied in multi-intersection traffic signal control. The MDRL algorithms produce the decentralized cooperative traffic-signal policies via specialized multi-agent settings in certain traffic networks. However, the state-of-the-art MDRL algorithms seem to have some drawbacks. (1) It is desirable that the traffic-signal policies can be smoothly transferred to diverse traffic networks, however, the adopted specialized multi-agent settings hinder the traffic-signal policies to transfer and generalize to new traffic networks. (2) Existing MDRL algorithms which are based on deep neural networks cannot flexibly tackle a time-varying number of vehicles traversing the traffic networks. (3) Existing MDRL algorithms which are based on homogeneous graph neural networks fail to capture the heterogeneous features of objects in traffic networks. Motivated by the above observations, in this paper, we propose an algorithm, referred to as Inductive Heterogeneous Graph Multi-agent Actor-critic (IHG-MA) algorithm, for multi-intersection traffic signal control. The proposed IHG-MA algorithm has two features: (1) It conducts representation learning using a proposed inductive heterogeneous graph neural network (IHG), which is an inductive algorithm. The proposed IHG algorithm can generate embeddings for previously unseen nodes (e.g., new entry vehicles) and new graphs (e.g., new traffic networks). But unlike the algorithms based on the homogeneous graph neural network, IHG algorithm not only encodes heterogeneous features of each node, but also encodes heterogeneous structural (graph) information. (2) It also conducts policy learning using a proposed multi-agent actor-critic(MA), which is a decentralized cooperative framework. The proposed MA framework employs the final embeddings to compute the Q-value and policy, and then optimizes the whole algorithm via the Q-value and policy loss. Experimental results on different traffic datasets illustrate that IHG-MA algorithm outperforms the state-of-the-art algorithms in terms of multiple traffic metrics, which seems to be a new promising algorithm for multi-intersection traffic signal control.

Hierarchical Graph Multi-agent Reinforcement Learning for Traffic Signal Control

TraCo: Learning Virtual Traffic Coordinator for Cooperation with Multi-Agent Reinforcement Learning.

Network Clustering-Based Multi-Agent Reinforcement Learning for Large-Scale Traffic Signal Control

Learning Decentralized Traffic Signal Controllers with Multi-Agent Graph Reinforcement Learning

Multi-agent Deep Reinforcement Learning collaborative Traffic Signal Control method considering intersection heterogeneity

Learning Multi-intersection Traffic Signal Control Via Coevolutionary Multi-Agent Reinforcement Learning

IHG-MA: Inductive heterogeneous graph multi-agent reinforcement learning for multi-intersection traffic signal control

AGRCNet: communicate by attentional graph relations in multi-agent reinforcement learning for traffic signal control

Multi-Agent Reinforcement Learning Based on Representational Communication for Large-Scale Traffic Signal Control

Multi-Agent Deep Reinforcement Learning for Urban Traffic Light Control in Vehicular Networks

Graph cooperation deep reinforcement learning for ecological urban traffic signal control

Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control

Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphs

Extensible Hierarchical Multi-Agent Reinforcement-Learning Algorithm in Traffic Signal Control

Heterogeneous Multi-Agent Reinforcement Learning Based on Adaptive Curiosity for Traffic Signal Control

An Inductive Heterogeneous Graph Attention-Based Multi-Agent Deep Graph Infomax Algorithm for Adaptive Traffic Signal Control

Feudal Multi-Agent Reinforcement Learning with Adaptive Network Partition for Traffic Signal Control

Neighborhood Cooperative Multiagent Reinforcement Learning for Adaptive Traffic Signal Control in Epidemic Regions

Applications in Traffic Signal Control: A Distributed Policy Gradient Decomposition Algorithm

Dynamic Traffic Signal Control Using Mean Field Multi‐agent Reinforcement Learning in Large Scale Road‐networks