Abstract:The vehicle platoon will be the most dominant driving mode on future roads. To the best of our knowledge, few reinforcement learning (RL) algorithms have been applied in vehicle platoon control, which has large-scale action and state spaces. Some RL-based methods were applied to solve single-agent problems. If we need to tackle multiagent problems, we will use multiagent RL algorithms since the parameters space grows exponentially with the increasing number of agents involved. Previous multiagent RL algorithms generally may provide redundant information to agents, indicating a large amount of useless or unrelated information, which may cause to be difficult for convergence training and pattern extractions from shared information. Also, random actions usually contribute to crashes, especially at the beginning of training. In this study, a communication proximal policy optimization (CommPPO) algorithm was proposed to tackle the above issues. In specific, the CommPPO model adopts a parameter-sharing structure to allow the dynamic variation of agent numbers, which can well handle various platoon dynamics, including splitting and merging. The communication protocol of the CommPPO consists of two parts. In the state part, the widely used predecessor-leader follower typology in the platoon is adopted to transmit global and local state information to agents. In the reward part, a new reward communication channel is proposed to solve the spurious reward and "lazy agent" problems in some existing multiagent RLs. Moreover, a curriculum learning approach is adopted to reduce crashes and speed up training. To validate the proposed strategy for platoon control, two existing multiagent RLs and a traditional platoon control strategy were applied in the same scenarios for comparison. Results showed that the CommPPO algorithm gained more rewards and achieved the largest fuel consumption reduction (11.6%).

Cooperative MARL-PPO Approach for Automated Highway Platoon Merging

TraCo: Learning Virtual Traffic Coordinator for Cooperation with Multi-Agent Reinforcement Learning.

Adaptive Vehicle Platooning with Joint Network-Traffic Approach

Merging Maneuver Technique for an Adaptive Autonomous Platoon in Multi-Lane Highways

Communication-Efficient Cooperative Multi-Agent PPO via Regulated Segment Mixture in Internet of Vehicles

Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic

Multi-Vehicle Cooperative Decision-Making in Merging Area Based on Deep Multi-Agent Reinforcement Learning

MARP: A Cooperative Multiagent DRL System for Connected Autonomous Vehicle Platooning

MAPPO-PIS: A Multi-Agent Proximal Policy Optimization Method with Prior Intent Sharing for CAVs' Cooperative Decision-Making

Cooperative Merging Strategy in Mixed Traffic Based on Optimal Final-State Phase Diagram with Flexible Highway Merging Points

A Reinforcement Learning-Based Vehicle Platoon Control Strategy for Reducing Energy Consumption in Traffic Oscillations

Combat Urban Congestion via Collaboration: Heterogeneous GNN-based MARL for Coordinated Platooning and Traffic Signal Control

Cooperative Highway Work Zone Merge Control Based on Reinforcement Learning in a Connected and Automated Environment

Platoon Merging Approach Based on Hybrid Trajectory Planning and CACC Strategies

Distributed multilane merging for connected autonomous vehicle platooning

Deep Reinforcement Learning for Multi-Objective Resource Allocation in Multi-Platoon Cooperative Vehicular Networks

iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning

Predictive Information Multiagent Deep Reinforcement Learning for Automated Truck Platooning Control

Cooperative Merging Strategy for Connected Vehicles at Highway On-Ramps

A Cooperative Truck Platooning Approach on Ramp Merging Area of Highway in Mixed Traffic

Safe and Efficient Cooperative Platooning