Abstract:As urban traffic condition is diverse and complicated, applying reinforcement learning to reduce traffic congestion becomes one of the hot and promising topics. Especially, how to coordinate the traffic light controllers of multiple intersections is a key challenge for multi-agent reinforcement learning (MARL). Most existing MARL studies are based on traditional $Q$-learning, but unstable environment leads to poor learning in the complicated and dynamic traffic scenarios. In this paper, we propose a novel multi-agent recurrent deep deterministic policy gradient (MARDDPG) algorithm based on deep deterministic policy gradient (DDPG) algorithm for traffic light control (TLC) in vehiclar networks. Specifically, the centralized learning in each critic network enables each agent to estimate the policies of other agents in the decision-making process and each agent can coordinate with each other, alleviating the problem of poor learning performance caused by environmental instability. The decentralized execution enables each agent to make decisions independently. We share parameters in actor networks to speed up the training process and reduce the memory footprint. The addition of LSTM is beneficial to alleviate the instability of the environment caused by partial observable state. We utilize surveillance cameras and vehicular networks to collect status information for each intersection. Unlike previous work, we have not only considered the vehicle but also considered the pedestrians waiting to pass through the intersection. Moreover, we also set different priorities for buses and ordinary vehicles. The experimental results in a vehicular network show that our method can run stably in various scenarios and coordinate multiple intersections, which significantly reduces vehicle congestion and pedestrian congestion.

Fast Multi-Class Vehicle Cooperative Path Optimization in Complex Urban V2X Transportation: A Novel Parallel Multi-Agent Reinforcement Learning Approach

Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

Cooperative Path Planning with Asynchronous Multiagent Reinforcement Learning

Multi-agent Path Finding for Mixed Autonomy Traffic Coordination

Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach

Multi-agent Path Finding for Cooperative Autonomous Driving

Cooperative Optimization of Traffic Signals and Vehicle Speed Using a Novel Multi-agent Deep Reinforcement Learning

Population Game-Assisted Multi-Agent Reinforcement Learning Method for Dynamic Multi-Vehicle Route Selection

A Multi-Agent Reinforcement Learning Approach For Safe and Efficient Behavior Planning Of Connected Autonomous Vehicles

Online longitudinal trajectory planning for connected and autonomous vehicles in mixed traffic flow with deep reinforcement learning approach

Joint Optimization of Traffic Signal Control and Vehicle Routing in Signalized Road Networks using Multi-Agent Deep Reinforcement Learning

Dynamic Queue-Jump Lane for Emergency Vehicles under Partially Connected Settings: A Multi-Agent Deep Reinforcement Learning Approach

MARP: A Cooperative Multiagent DRL System for Connected Autonomous Vehicle Platooning

Multi-agent policy learning-based path planning for autonomous mobile robots

Towards Cooperative Driving among Heterogeneous CAVs: A Safe Multi-Agent Reinforcement Learning Approach

Optimizing Traffic Lights with Multi-agent Deep Reinforcement Learning and V2X communication

Multi-agent reinforcement learning-based dynamic task assignment for vehicles in urban transportation system

Cooperative Multiagent Deep Deterministic Policy Gradient (comaddpg) for Intelligent Connected Transportation with Unsignalized Intersection

Routing optimization with Monte Carlo Tree Search-based multi-agent reinforcement learning

Multi-Agent Deep Reinforcement Learning for Urban Traffic Light Control in Vehicular Networks

Multi-agent Reinforcement Learning for Vehicular Task Offloading with Multi-Step Trajectory Prediction