Abstract:In this thesis, I propose a family of fully decentralized deep multi-agent reinforcement learning (MARL) algorithms to achieve high, real-time performance in network-level traffic signal control. In this approach, each intersection is modeled as an agent that plays a Markovian Game against the other intersection nodes in a traffic signal network modeled as an undirected graph, to approach the optimal reduction in delay. Following Partially Observable Markov Decision Processes (POMDPs), there are 3 levels of communication schemes between adjacent learning agents: independent deep Q-leaning (IDQL), shared states reinforcement learning (S2RL) and a shared states & rewards version of S2RL--S2R2L. In these 3 variants of decentralized MARL schemes, individual agent trains its local deep Q network (DQN) separately, enhanced by convergence-guaranteed techniques like double DQN, prioritized experience replay, multi-step bootstrapping, etc. To test the performance of the proposed three MARL algorithms, a SUMO-based simulation platform is developed to mimic the traffic evolution of the real world. Fed with random traffic demand between permitted OD pairs, a 4x4 Manhattan-style grid network is set up as the testbed, two different vehicle arrival rates are generated for model training and testing. The experiment results show that S2R2L has a quicker convergence rate and better convergent performance than IDQL and S2RL in the training process. Moreover, three MARL schemes all reveal exceptional generalization abilities. Their testing results surpass the benchmark Max Pressure (MP) algorithm, under the criteria of average vehicle delay, network-level queue length and fuel consumption rate. Notably, S2R2L has the best testing performance of reducing 34.55% traffic delay and dissipating 10.91% queue length compared with MP.

Distributed Optimization of Regional Traffic Signals via Deep Reinforcement Learning

Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

A distributed deep reinforcement learning method for traffic light control

Adaptive Traffic Signal Control Model on Intersections Based on Deep Reinforcement Learning

A multi‐agent deep reinforcement learning approach for traffic signal coordination

Joint Optimization of Traffic Signal Control and Vehicle Routing in Signalized Road Networks using Multi-Agent Deep Reinforcement Learning

Carbon Dioxide Emission Reduction-Oriented Optimal Control of Traffic Signals in Mixed Traffic Flow Based on Deep Reinforcement Learning

Optimizing Traffic Lights with Multi-agent Deep Reinforcement Learning and V2X communication

Urban Traffic Control in Software Defined Internet of Things via a Multi-Agent Deep Reinforcement Learning Approach

Decentralized Deep Reinforcement Learning for Network Level Traffic Signal Control

Cooperative Multi-Objective Reinforcement Learning for Traffic Signal Control and Carbon Emission Reduction

Cooperative Reinforcement Learning on Traffic Signal Control

Cooperative Optimization of Traffic Signals and Vehicle Speed Using a Novel Multi-agent Deep Reinforcement Learning

Integrating independent and centralized multi-agent reinforcement learning for traffic signal network optimization

A Novel Multi-Agent Deep RL Approach for Traffic Signal Control

Mean Field Multi-Agent Reinforcement Learning Method for Area Traffic Signal Control

Two-layer Coordinated Reinforcement Learning for Traffic Signal Control in Traffic Network

Research on Intelligent Signal Timing Optimization of Signalized Intersection Based on Deep Reinforcement Learning Using Floating Car Data