Abstract:In this article, we improve the proximal policy optimisation (PPO) algorithm in deep reinforcement learning and propose an MSA‐PPO algorithm, which adopts the self‐attention mechanism for input data processing, effectively identifies and focuses on the most important information in the interaction between vehicles to improve the overall performance of the system, in addition to applying the ineffective action masking mechanism to select the effective actions under specific conditions and narrow the decision space, greatly improve the learning efficiency, and thus improve the overall performance of the system. The deployment of autonomous vehicles (AVs) in complex urban environments faces numerous challenges, especially at intersections where they coexist with human‐driven vehicles (HVs), resulting in increased safety risks. In response, this study proposes an improved control strategy based on the Proximal Policy Optimization (PPO) algorithm, specifically designed for hybrid intersections, known as MSA‐PPO. First, the Self‐Attention Mechanism (SAM) is introduced into the algorithmic framework to quickly identify the surrounding vehicles with a greater impact on the ego vehicle from different perspectives, accelerating data processing and improving decision quality. Second, an invalid action masking mechanism is adopted to reduce the action space, ensuring actions are only selected from feasible sets, thereby enhancing decision efficiency. Finally, comparative and ablation experiments in hybrid intersection simulation environments of varying complexity are conducted to validate the algorithm's effectiveness. The results show that the improved algorithm converges faster, achieves higher decision accuracy, and demonstrates the highest speed levels during driving compared to other baseline algorithms.

Intelligent Traffic Signal Control with Deep Reinforcement Learning at Single Intersection

Q-learning Based Traffic Signal Control Method for an Isolated Intersection

Distributed Optimization of Regional Traffic Signals via Deep Reinforcement Learning

Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning

Adaptive Traffic Signal Control Model on Intersections Based on Deep Reinforcement Learning

Intelligent Traffic Light via Policy-based Deep Reinforcement Learning

Deep Reinforcement Learning for Intersection Signal Control Considering Pedestrian Behavior

Pri-DDQN: Learning Adaptive Traffic Signal Control Strategy Through a Hybrid Agent

Adaptive Optimization of Traffic Signal Timing Via Deep Reinforcement Learning

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

An Intelligent Traffic Signal Control System Based on Deep Reinforcement Learning

Multi-Agent Reinforcement Learning-Based Autonomous Intersection Management Protocol with Attention Mechanism

Adaptive Dynamic Programming For Multi-Intersections Traffic Signal Intelligent Control

Proximal Policy Optimization Through a Deep Reinforcement Learning Framework for Multiple Autonomous Vehicles at a Non-Signalized Intersection

Intersection decision making for autonomous vehicles based on improved PPO algorithm

Centralized Cooperation for Connected and Automated Vehicles at Intersections by Proximal Policy Optimization

Urban Traffic Control in Software Defined Internet of Things via a Multi-Agent Deep Reinforcement Learning Approach

Traffic Signal Optimization for Multiple Intersections Based on Reinforcement Learning

Eco-driving for Intelligent Electric Vehicles at Signalized Intersection: A Proximal Policy Optimization Approach

A Reinforcement Learning Approach for Intelligent Traffic Signal Control at Urban Intersections