Abstract:<p>A connected autonomous vehicle (CAV) network can be defined as a set of connected vehicles including CAVs that operate on a specific spatial scope that may be a road network, corridor, or segment. The spatial scope constitutes an environment where traffic information is shared and instructions are issued for controlling the CAVs movements. Within such a spatial scope, high-level cooperation among CAVs fostered by joint planning and control of their movements can greatly enhance the safety and mobility performance of their operations. Unfortunately, the highly combinatory and volatile nature of CAV networks due to the dynamic number of agents (vehicles) and the fast-growing joint action space associated with multi-agent driving tasks pose difficultly in achieving cooperative control. The problem is NP-hard and cannot be efficiently resolved using rule-based control techniques. Also, there is a great deal of information in the literature regarding sensing technologies and control logic in CAV operations but relatively little information on the integration of information from collaborative sensing and connectivity sources. Therefore, we present a novel deep reinforcement learning-based algorithm that combines graphic convolution neural network with deep Q-network to form an innovative graphic convolution Q network that serves as the information fusion module and decision processor. In this study, the spatial scope we consider for the CAV network is a multi-lane road corridor. We demonstrate the proposed control algorithm using the application context of freeway lane-changing at the approaches to an exit ramp. For purposes of comparison, the proposed model is evaluated vis-à-vis traditional rule-based and long short-term memory-based fusion models. The results suggest that the proposed model is capable of aggregating information received from sensing and connectivity sources and prescribing efficient operative lane-change decisions for multiple CAVs, in a manner that enhances safety and mobility. That way, the operational intentions of individual CAVs can be fulfilled even in partially observed and highly dynamic mixed traffic streams. The paper presents experimental evidence to demonstrate that the proposed algorithm can significantly enhance CAV operations. The proposed algorithm can be deployed at roadside units or cloud platforms or other centralized control facilities.</p>

An Inductive Heterogeneous Graph Attention-Based Multi-Agent Deep Graph Infomax Algorithm for Adaptive Traffic Signal Control

IHG-MA: Inductive heterogeneous graph multi-agent reinforcement learning for multi-intersection traffic signal control

Hierarchical Graph Multi-agent Reinforcement Learning for Traffic Signal Control

Learning Multiaspect Traffic Couplings by Multirelational Graph Attention Networks for Traffic Prediction

Multi-agent Deep Reinforcement Learning collaborative Traffic Signal Control method considering intersection heterogeneity

A large-scale traffic signal control algorithm based on multi-layer graph deep reinforcement learning

Enhancing traffic signal control with composite deep intelligence

Adaptive Traffic Signal Control Based on Graph Neural Networks and Dynamic Entropy-Constrained Soft Actor–Critic

AGRCNet: communicate by attentional graph relations in multi-agent reinforcement learning for traffic signal control

Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control

Cooperative Multi-Agent Actor–critic Control of Traffic Network Flow Based on Edge Computing

DynSTGAT: Dynamic Spatial-Temporal Graph Attention Network for Traffic Signal Control

Multi-Agent Attention Double Actor-Critic Framework for Intelligent Traffic Light Control in Urban Scenarios With Hybrid Traffic

Graph cooperation deep reinforcement learning for ecological urban traffic signal control

Multi-Agent Transfer Reinforcement Learning With Multi-View Encoder for Adaptive Traffic Signal Control

Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphs

Learning scalable multi-agent coordination by spatial differentiation for traffic signal control

Combat Urban Congestion via Collaboration: Heterogeneous GNN-based MARL for Coordinated Platooning and Traffic Signal Control

Learning Decentralized Traffic Signal Controllers with Multi-Agent Graph Reinforcement Learning

Graph neural network and reinforcement learning for multi‐agent cooperative control of connected autonomous vehicles