A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Network

Jiqian Dong,Sikai Chen,Paul Young Joun Ha,Yujie Li,Samuel Labi

DOI: https://doi.org/10.48550/arXiv.2010.05437

2020-10-12

Abstract:Connected Autonomous Vehicle (CAV) Network can be defined as a collection of CAVs operating at different locations on a multilane corridor, which provides a platform to facilitate the dissemination of operational information as well as control instructions. Cooperation is crucial in CAV operating systems since it can greatly enhance operation in terms of safety and mobility, and high-level cooperation between CAVs can be expected by jointly plan and control within CAV network. However, due to the highly dynamic and combinatory nature such as dynamic number of agents (CAVs) and exponentially growing joint action space in a multiagent driving task, achieving cooperative control is NP hard and cannot be governed by any simple rule-based methods. In addition, existing literature contains abundant information on autonomous driving's sensing technology and control logic but relatively little guidance on how to fuse the information acquired from collaborative sensing and build decision processor on top of fused information. In this paper, a novel Deep Reinforcement Learning (DRL) based approach combining Graphic Convolution Neural Network (GCN) and Deep Q Network (DQN), namely Graphic Convolution Q network (GCQ) is proposed as the information fusion module and decision processor. The proposed model can aggregate the information acquired from collaborative sensing and output safe and cooperative lane changing decisions for multiple CAVs so that individual intention can be satisfied even under a highly dynamic and partially observed mixed traffic. The proposed algorithm can be deployed on centralized control infrastructures such as road-side units (RSU) or cloud platforms to improve the CAV operation.

Artificial Intelligence,Systems and Control

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to achieve multi - agent collaborative control in the Connected Autonomous Vehicle (CAV) network. Specifically, the paper focuses on how to achieve efficient and safe collaborative control in a dynamically changing number of agents (i.e., CAVs) and an exponentially growing joint action space, which is an NP - hard problem and difficult to handle with traditional rule - based methods. In addition, although the existing literature provides abundant information on autonomous driving perception technologies and control logics, it offers less guidance on how to fuse the information obtained from collaborative perception and build a decision - making processor on this basis. To this end, the paper proposes a new method based on Deep Reinforcement Learning (DRL), which combines Graph Convolutional Neural Network (GCN) and Deep Q - Network (DQN), namely Graph Convolutional Q - Network (GCQ). This model aims to aggregate the information obtained from collaborative perception and output safe and cooperative lane - changing decisions for multiple CAVs, which can also meet individual intentions even in highly dynamic and partially observed mixed traffic. The main contributions of the paper are as follows: 1. Use a graphical representation to describe the mixed - traffic driving environment and the information flow topology. 2. Develop a GCN - based information fusion module to promote communication and cooperation between vehicles. 3. Combine GCN and DQN into an end - to - end multi - agent decision - making processor for controlling the lane - changing decisions of CAVs on road segments. Through this method, the paper aims to overcome the challenges brought by a dynamically changing number of agents and provide an effective solution to improve the safety of CAV network operations and the overall mobility of the system.

A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Network

Graph neural network and reinforcement learning for multi‐agent cooperative control of connected autonomous vehicles

Multi-View Graph Convolution Network Reinforcement Learning for CAVs Cooperative Control in Highway Mixed Traffic

Adaptive Traffic Signal Control for Large-Scale Scenario with Cooperative Group-based Multi-agent Reinforcement Learning

Coordination for Connected and Automated Vehicles at Non-Signalized Intersections: A Value Decomposition-Based Multiagent Deep Reinforcement Learning Approach

Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion

Communication-Efficient Decentralized Multi-Agent Reinforcement Learning for Cooperative Adaptive Cruise Control

Heuristic-Based Multi-Agent Deep Reinforcement Learning Approach for Coordinating Connected and Automated Vehicles at Non-Signalized Intersection

A Multi-Agent Reinforcement Learning Approach For Safe and Efficient Behavior Planning Of Connected Autonomous Vehicles

Safety Guaranteed Robust Multi-Agent Reinforcement Learning with Hierarchical Control for Connected and Automated Vehicles

A Multi-Agent Deep Reinforcement Learning Coordination Framework for Connected and Automated Vehicles at Merging Roadways

Managing Mixed Traffic at Signalized Intersections: an Adaptive Signal Control and CAV Coordination System Based on Deep Reinforcement Learning.

Cooperative Multi-Agent Actor–critic Control of Traffic Network Flow Based on Edge Computing

CCGN: Centralized collaborative graphical transformer multi-agent reinforcement learning for multi-intersection signal free-corridor

A multi-agent reinforcement learning-based longitudinal and lateral control of CAVs to improve traffic efficiency in a mandatory lane change scenario

Cooperative Optimization of Traffic Signals and Vehicle Speed Using a Novel Multi-agent Deep Reinforcement Learning

Real-time Cooperative Vehicle Coordination at Unsignalized Road Intersections

Dynamic Queue-Jump Lane for Emergency Vehicles under Partially Connected Settings: A Multi-Agent Deep Reinforcement Learning Approach

Shared Information-Based Safe And Efficient Behavior Planning For Connected Autonomous Vehicles

Distributed Cooperative Control and Optimization of Connected Automated Vehicles Platoon Against Cut-in Behaviors of Social Drivers

Reasoning graph-based reinforcement learning to cooperate mixed connected and autonomous traffic at unsignalized intersections