Abstract:Connected automated vehicles (CAVs) are broadly recognized as next-generation transformative transportation technologies having great potential to improve traffic safety, efficiency, and stability. Efficiently controlling CAVs on two-dimensional curvilinear roadways to follow preceding vehicles is denoted as the two-dimensional car-following process, which is highly critical; this process is challenging to implement owing to the complexity and varied nature of driving environments. This study proposes an innovative integrated two-dimensional control strategy for CAVs based on deep reinforcement learning (DRL), which efficiently regulates the two-dimensional car-following process of CAVs in terms of both stability-wise longitudinal control performance and accurate lateral path-tracking performance. Within the control framework, each CAV can receive the surrounding information from downstream vehicles and roadway geometry based on vehicle-to-everything (V2X) communication. To better utilize this information, we designed a physics-informed DRL state fusion approach and reward function, which efficiently embeds prior physics knowledge and borrows the merits of the equilibrium and consensus concepts from the control theory. Given the physics-informed information, the DRL-based controller outputs the integrated control instructions for both longitudinal and lateral control. For training, we constructed a roadway with a set of varying curvatures and embedded the ground-truth vehicle trajectory datasets to more effectively capture the realistic variations in the roadway geometry and driving environment. To facilitate value function approximation and enhance the policy iteration process in training, the distributed proximal policy optimization (DPPO) algorithm was applied, owing to its balanced performance. A series of simulated experiments were conducted to validate the controller’s lateral control accuracy and stability-wise oscillation dampening performance in diverse traffic scenarios, including extreme ones.

Distributed Connected Automated Vehicles Control under Real‐Time Aggregated Macroscopic Car Following Behavior Estimation Based on Deep Reinforcement Learning

A Deep Reinforcement Learning Based Distributed Control Strategy for Connected Automated Vehicles in Mixed Traffic Platoon

Connected Automated Vehicle Cooperative Control with a Deep Reinforcement Learning Approach in a Mixed Traffic Environment

Partially Connected Automated Vehicle Cooperative Control Strategy with a Deep Reinforcement Learning Approach

Deep Reinforcement Learning Car-Following Control Based on Multivehicle Motion Prediction

Multi-Agent DRL-Controlled Connected and Automated Vehicles in Mixed Traffic with Time Delays

Physics-informed Deep Reinforcement Learning-Based Integrated Two-Dimensional Car-Following Control Strategy for Connected Automated Vehicles

A Machine Learning Method for Dynamic Traffic Control and Guidance on Freeway Networks

Augmented Mixed Vehicular Platoon Control with Dense Communication Reinforcement Learning for Traffic Oscillation Alleviation

Hybrid car following control for CAVs: Integrating linear feedback and deep reinforcement learning to stabilize mixed traffic

Longitudinal control of connected and automated vehicles among signalized intersections in mixed traffic flow with deep reinforcement learning approach

CAV-AHDV-CAV: Mitigating Traffic Oscillations for CAVs through a Novel Car-Following Structure and Reinforcement Learning

Enhancing System-Level Safety in Mixed-Autonomy Platoon via Safe Reinforcement Learning

Altruistic cooperative adaptive cruise control of mixed traffic platoon based on deep reinforcement learning

Multi-agent reinforcement learning for ecological car-following control in mixed traffic

Dampen the Stop-and-Go Traffic with Connected and Automated Vehicles -- A Deep Reinforcement Learning Approach

Safety Guaranteed Robust Multi-Agent Reinforcement Learning with Hierarchical Control for Connected and Automated Vehicles

Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion

Managing Mixed Traffic at Signalized Intersections: an Adaptive Signal Control and CAV Coordination System Based on Deep Reinforcement Learning.

Longitudinal control of automated vehicles: A novel approach by integrating deep reinforcement learning with intelligent driver model

A Multi-agent Reinforcement Learning Based Control Method for Connected and Autonomous Vehicles in A Mixed Platoon