Abstract:Traffic signal control (TSC) plays a crucial role in enhancing traffic capacity. In recent years, researchers have demonstrated improved performance by utilizing deep reinforcement learning (DRL) for optimizing TSC. However, existing DRL frameworks predominantly rely on manually crafted states, actions, and reward designs, which limit direct information exchange between the DRL agent and the environment. To overcome this challenge, we propose a novel design method that maintains consistency among states, actions, and rewards, named uniformity state-action-reward (USAR) method for TSC. The USAR method relies on: 1) Updating the action selection for the next time step using a formula based on the state perceived by the agent at the current time step, thereby encouraging rapid convergence to the optimal strategy from state perception to action; and 2) integrating the state representation with the reward function design, allowing for precise assessment of the efficacy of past action strategies based on the received feedback rewards. The consistency-preserving design method jointly optimizes the TSC strategy through the updates and feedback among the Markov elements. Furthermore, the method proposed in this paper employs a residual block into the DRL model. It introduces an additional pathway between the input and output layers to transfer feature information, thus promoting the flow of information across different network layers. To assess the effectiveness of our approach, we conducted a series of simulation experiments using the simulation of urban mobility. The USAR method, incorporating a residual block, outperformed other methods and exhibited the best performance in several evaluation metrics.

Comparing Observation and Action Representations for Deep Reinforcement Learning in $μ$RTS

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

S2RL: DoWe Really Need to Perceive All States in Deep Multi-Agent Reinforcement Learning?

S2RL: Do We Really Need to Perceive All States in Deep Multi-Agent Reinforcement Learning?

S2rl

Reinforcement actor-critic learning as a rehearsal in MicroRTS

Investigating the Impact of Choice on Deep Reinforcement Learning for Space Controls

Comparing Reinforcement Learning and Human Learning using the Game of Hidden Rules

On Improving Deep Reinforcement Learning for POMDPs

Localized Observation Abstraction Using Piecewise Linear Spatial Decay for Reinforcement Learning in Combat Simulations

A Survey of Deep Reinforcement Learning in Video Games

Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

Playing Card-Based RTS Games with Deep Reinforcement Learning.

Research on Action Strategies and Simulations of DRL and MCTS-based Intelligent Round Game

Dynamic Observation Policies in Observation Cost-Sensitive Reinforcement Learning

Exploring Exploration: Comparing Children with RL Agents in Unified Environments

A Comparative Study of Deep Reinforcement Learning Models: DQN vs PPO vs A2C

Algorithms or Actions? A Study in Large-Scale Reinforcement Learning

Dealing with uncertainty: Balancing exploration and exploitation in deep recurrent reinforcement learning

Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain