Abstract:Most of the existing traffic signal control strategies are hard to satisfy the real-time requirements of traffic big data analysis, knowledge reasoning and decision making for sophisticated traffic dynamics and heterogeneous intersection structures in the context of Internet of Vehicles (IoV). In this paper, we attempt to propose a cooperative multi-agent actor–critic (CMAC) deep reinforcement learning (DRL) approach with value decomposition based on edge computing architecture. The intuition behind CMAC is to decompose the global actor–critic learning tasks into several local actor–critic sub-problems with respect to each intersection. Each agent searches the local optimal decision by actor–critic network that takes the discrete state encoding about several consecutive frames of image-like traffic states as the inputs of the network. Among them, the green ratio output strategy considering multiple constraints is formulated in the output layer of the actor network, so that the continuous control of traffic signals using multi-agent DRL (MADRL) can be realized. Furthermore, a cooperative mechanism that considers contribution weight distributions of local agents to the global traffic pattern is proposed to coordinate multiple local agents to evolve toward global optimization. Especially, some parallel training tasks of CMAC with a large number of computing loads are deployed on the cloud side in the edge computing architecture to accelerate learning and reconstructing knowledge. The well-trained multi-agent model is downloaded from the cloud side into the edge side for real-time decision making of traffic network flow adaptive control. Simulation results with regard to a realistic traffic network demonstrate that the proposed CMAC approach under edge computing architecture outperforms the value-decomposition based multi-agent actor–critic (VMAC), independent multi-agent actor–critic (IMAC), and the fixed timing control (FTC) in terms of alleviating traffic congestion.

Mastering Arterial Traffic Signal Control with Multi-Agent Attention-Based Soft Actor-Critic Model

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

TraCo: Learning Virtual Traffic Coordinator for Cooperation with Multi-Agent Reinforcement Learning.

Multi-Agent Transfer Reinforcement Learning With Multi-View Encoder for Adaptive Traffic Signal Control

Multi-Agent Attention Double Actor-Critic Framework for Intelligent Traffic Light Control in Urban Scenarios With Hybrid Traffic

Cooperative traffic signal control through a counterfactual multi-agent deep actor critic approach

AGRCNet: communicate by attentional graph relations in multi-agent reinforcement learning for traffic signal control

Mean Field Multi-Agent Reinforcement Learning Method for Area Traffic Signal Control

Network Clustering-Based Multi-Agent Reinforcement Learning for Large-Scale Traffic Signal Control

Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphs

Cooperative Multi-Agent Actor–critic Control of Traffic Network Flow Based on Edge Computing

Distributed Signal Control of Arterial Corridors Using Multi-Agent Deep Reinforcement Learning

Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control

MaCAR: Urban Traffic Light Control Via Active Multi-Agent Communication and Action Rectification

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

Urban Traffic Light Control Via Active Multi-agent Communication and Supply-Demand Modeling

Multi-agent Deep Reinforcement Learning collaborative Traffic Signal Control method considering intersection heterogeneity

Fairness-aware multi-agent reinforcement learning and visual perception for adaptive traffic signal control

A multi-agent reinforcement learning method with curriculum transfer for large-scale dynamic traffic signal control

Feudal Multi-Agent Reinforcement Learning with Adaptive Network Partition for Traffic Signal Control

A Novel Multi-Agent Deep RL Approach for Traffic Signal Control