Abstract:As urban traffic condition is diverse and complicated, applying reinforcement learning to reduce traffic congestion becomes one of the hot and promising topics. Especially, how to coordinate the traffic light controllers of multiple intersections is a key challenge for multi-agent reinforcement learning (MARL). Most existing MARL studies are based on traditional $Q$-learning, but unstable environment leads to poor learning in the complicated and dynamic traffic scenarios. In this paper, we propose a novel multi-agent recurrent deep deterministic policy gradient (MARDDPG) algorithm based on deep deterministic policy gradient (DDPG) algorithm for traffic light control (TLC) in vehiclar networks. Specifically, the centralized learning in each critic network enables each agent to estimate the policies of other agents in the decision-making process and each agent can coordinate with each other, alleviating the problem of poor learning performance caused by environmental instability. The decentralized execution enables each agent to make decisions independently. We share parameters in actor networks to speed up the training process and reduce the memory footprint. The addition of LSTM is beneficial to alleviate the instability of the environment caused by partial observable state. We utilize surveillance cameras and vehicular networks to collect status information for each intersection. Unlike previous work, we have not only considered the vehicle but also considered the pedestrians waiting to pass through the intersection. Moreover, we also set different priorities for buses and ordinary vehicles. The experimental results in a vehicular network show that our method can run stably in various scenarios and coordinate multiple intersections, which significantly reduces vehicle congestion and pedestrian congestion.

Efficient Policy Transfer in Large-Scale Traffic Light Control Via Multi-Agent Hierarchical Reinforcement Learning

TraCo: Learning Virtual Traffic Coordinator for Cooperation with Multi-Agent Reinforcement Learning.

Network Clustering-Based Multi-Agent Reinforcement Learning for Large-Scale Traffic Signal Control

A multi-agent reinforcement learning method with curriculum transfer for large-scale dynamic traffic signal control

Multi-Agent Deep Reinforcement Learning for Urban Traffic Light Control in Vehicular Networks

Multi-Agent Reinforcement Learning Based on Representational Communication for Large-Scale Traffic Signal Control

Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control

Large-Scale Traffic Signal Control Using a Novel Multiagent Reinforcement Learning

Integrating independent and centralized multi-agent reinforcement learning for traffic signal network optimization

STMARL: A Spatio-Temporal Multi-Agent Reinforcement Learning Approach for Cooperative Traffic Light Control

Learning Decentralized Traffic Signal Controllers with Multi-Agent Graph Reinforcement Learning

Toward A Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control

Multi-Agent Transfer Reinforcement Learning With Multi-View Encoder for Adaptive Traffic Signal Control

Traffic Signal Control Using a Cooperative EWMA-based Multi-Agent Reinforcement Learning.

Intelligent Traffic Light via Policy-based Deep Reinforcement Learning

Feudal Multi-Agent Reinforcement Learning with Adaptive Network Partition for Traffic Signal Control

MixLight: Mixed-Agent Cooperative Reinforcement Learning for Traffic Light Control

A multi-agent reinforcement learning based approach for intelligent traffic signal control

Adaptive Traffic Signal Control for Large-Scale Scenario with Cooperative Group-based Multi-agent Reinforcement Learning

X-Light: Cross-City Traffic Signal Control Using Transformer on Transformer as Meta Multi-Agent Reinforcement Learner