Abstract:As urban traffic condition is diverse and complicated, applying reinforcement learning to reduce traffic congestion becomes one of the hot and promising topics. Especially, how to coordinate the traffic light controllers of multiple intersections is a key challenge for multi-agent reinforcement learning (MARL). Most existing MARL studies are based on traditional $Q$-learning, but unstable environment leads to poor learning in the complicated and dynamic traffic scenarios. In this paper, we propose a novel multi-agent recurrent deep deterministic policy gradient (MARDDPG) algorithm based on deep deterministic policy gradient (DDPG) algorithm for traffic light control (TLC) in vehiclar networks. Specifically, the centralized learning in each critic network enables each agent to estimate the policies of other agents in the decision-making process and each agent can coordinate with each other, alleviating the problem of poor learning performance caused by environmental instability. The decentralized execution enables each agent to make decisions independently. We share parameters in actor networks to speed up the training process and reduce the memory footprint. The addition of LSTM is beneficial to alleviate the instability of the environment caused by partial observable state. We utilize surveillance cameras and vehicular networks to collect status information for each intersection. Unlike previous work, we have not only considered the vehicle but also considered the pedestrians waiting to pass through the intersection. Moreover, we also set different priorities for buses and ordinary vehicles. The experimental results in a vehicular network show that our method can run stably in various scenarios and coordinate multiple intersections, which significantly reduces vehicle congestion and pedestrian congestion.

Urban Traffic Multi-Agent System Based on RMM and Bayesian Learning

Stochastic Model Predictive Control for Urban Traffic Networks

TraCo: Learning Virtual Traffic Coordinator for Cooperation with Multi-Agent Reinforcement Learning.

Mean Field Multi-Agent Reinforcement Learning Method for Area Traffic Signal Control

A Multi-Agent Coordinate Model for Urban Traffic Signal Control

Urban Traffic Light Control Via Active Multi-agent Communication and Supply-Demand Modeling

Network Clustering-Based Multi-Agent Reinforcement Learning for Large-Scale Traffic Signal Control

Cooperative traffic signal control through a counterfactual multi-agent deep actor critic approach

A Cooperative Multiagent System for Traffic Signal Control Using Game Theory and Reinforcement Learning

Multiobjective Reinforcement Learning for Traffic Signal Control Using Vehicular Ad Hoc Network

Multi-agent reinforcement learning for traffic congestion on one-way multi-lane highways

A multi-agent reinforcement learning based approach for intelligent traffic signal control

A multi‐agent deep reinforcement learning approach for traffic signal coordination

Batch-Augmented Multi-Agent Reinforcement Learning for Efficient Traffic Signal Optimization

Multi-Agent Deep Reinforcement Learning for Urban Traffic Light Control in Vehicular Networks

A multi-agent reinforcement learning method with curriculum transfer for large-scale dynamic traffic signal control

Optimizing multi-agent based urban traffic signal control system

Learning scalable multi-agent coordination by spatial differentiation for traffic signal control

An Information Fusion Approach to Intelligent Traffic Signal Control Using the Joint Methods of Multiagent Reinforcement Learning and Artificial Intelligence of Things

Multi-agent System Model for Urban Traffic Simulation and Optimizing Based on Random Walk

Multi-agent reinforcement learning-based dynamic task assignment for vehicles in urban transportation system