Abstract:Multi-agent deep reinforcement learning (MDRL) has been widely applied in multi-intersection traffic signal control. The MDRL algorithms produce the decentralized cooperative traffic-signal policies via specialized multi-agent settings in certain traffic networks. However, the state-of-the-art MDRL algorithms seem to have some drawbacks. (1) It is desirable that the traffic-signal policies can be smoothly transferred to diverse traffic networks, however, the adopted specialized multi-agent settings hinder the traffic-signal policies to transfer and generalize to new traffic networks. (2) Existing MDRL algorithms which are based on deep neural networks cannot flexibly tackle a time-varying number of vehicles traversing the traffic networks. (3) Existing MDRL algorithms which are based on homogeneous graph neural networks fail to capture the heterogeneous features of objects in traffic networks. Motivated by the above observations, in this paper, we propose an algorithm, referred to as Inductive Heterogeneous Graph Multi-agent Actor-critic (IHG-MA) algorithm, for multi-intersection traffic signal control. The proposed IHG-MA algorithm has two features: (1) It conducts representation learning using a proposed inductive heterogeneous graph neural network (IHG), which is an inductive algorithm. The proposed IHG algorithm can generate embeddings for previously unseen nodes (e.g., new entry vehicles) and new graphs (e.g., new traffic networks). But unlike the algorithms based on the homogeneous graph neural network, IHG algorithm not only encodes heterogeneous features of each node, but also encodes heterogeneous structural (graph) information. (2) It also conducts policy learning using a proposed multi-agent actor-critic(MA), which is a decentralized cooperative framework. The proposed MA framework employs the final embeddings to compute the Q-value and policy, and then optimizes the whole algorithm via the Q-value and policy loss. Experimental results on different traffic datasets illustrate that IHG-MA algorithm outperforms the state-of-the-art algorithms in terms of multiple traffic metrics, which seems to be a new promising algorithm for multi-intersection traffic signal control.

A Semi-Decentralized Feudal Multi-Agent Learned-Goal Algorithm for Multi-Intersection Traffic Signal Control.

Cooperative traffic signal control through a counterfactual multi-agent deep actor critic approach

A Meta Multi-agent Reinforcement Learning Algorithm for Multi-intersection Traffic Signal Control

Mean Field Multi-Agent Reinforcement Learning Method for Area Traffic Signal Control

Decentralized Deep Reinforcement Learning for Network Level Traffic Signal Control

Cooperative Traffic Signal Control Using Multi-step Return and Off-policy Asynchronous Advantage Actor-Critic Graph Algorithm.

An Information Fusion Approach to Intelligent Traffic Signal Control Using the Joint Methods of Multiagent Reinforcement Learning and Artificial Intelligence of Things

Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphs

Multi-agent Deep Reinforcement Learning collaborative Traffic Signal Control method considering intersection heterogeneity

A multi‐agent deep reinforcement learning approach for traffic signal coordination

Urban Traffic Light Control Via Active Multi-agent Communication and Supply-Demand Modeling

Multiobjective Reinforcement Learning for Traffic Signal Control Using Vehicular Ad Hoc Network

Multi-Agent Attention Double Actor-Critic Framework for Intelligent Traffic Light Control in Urban Scenarios With Hybrid Traffic

IHG-MA: Inductive heterogeneous graph multi-agent reinforcement learning for multi-intersection traffic signal control

Two-layer Coordinated Reinforcement Learning for Traffic Signal Control in Traffic Network

Feudal Multi-Agent Reinforcement Learning with Adaptive Network Partition for Traffic Signal Control

Adaptive Traffic Signal Control Model on Intersections Based on Deep Reinforcement Learning

Distributed Signal Control of Arterial Corridors Using Multi-Agent Deep Reinforcement Learning

Mastering Arterial Traffic Signal Control with Multi-Agent Attention-Based Soft Actor-Critic Model

Multi-Agent Learning Automata for Online Adaptive Control of Large-Scale Traffic Signal Systems

Cooperative Reinforcement Learning on Traffic Signal Control