Abstract:Connected automated vehicles (CAVs) are broadly recognized as next-generation transformative transportation technologies having great potential to improve traffic safety, efficiency, and stability. Efficiently controlling CAVs on two-dimensional curvilinear roadways to follow preceding vehicles is denoted as the two-dimensional car-following process, which is highly critical; this process is challenging to implement owing to the complexity and varied nature of driving environments. This study proposes an innovative integrated two-dimensional control strategy for CAVs based on deep reinforcement learning (DRL), which efficiently regulates the two-dimensional car-following process of CAVs in terms of both stability-wise longitudinal control performance and accurate lateral path-tracking performance. Within the control framework, each CAV can receive the surrounding information from downstream vehicles and roadway geometry based on vehicle-to-everything (V2X) communication. To better utilize this information, we designed a physics-informed DRL state fusion approach and reward function, which efficiently embeds prior physics knowledge and borrows the merits of the equilibrium and consensus concepts from the control theory. Given the physics-informed information, the DRL-based controller outputs the integrated control instructions for both longitudinal and lateral control. For training, we constructed a roadway with a set of varying curvatures and embedded the ground-truth vehicle trajectory datasets to more effectively capture the realistic variations in the roadway geometry and driving environment. To facilitate value function approximation and enhance the policy iteration process in training, the distributed proximal policy optimization (DPPO) algorithm was applied, owing to its balanced performance. A series of simulated experiments were conducted to validate the controller’s lateral control accuracy and stability-wise oscillation dampening performance in diverse traffic scenarios, including extreme ones.

Reinforcement Learning-Based Approach to Reduce Velocity Error in Car-Following for Autonomous Connected Vehicles

Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving

Deep Reinforcement Learning Car-Following Control Based on Multivehicle Motion Prediction

Velocity Control in Car-Following Behavior with Autonomous Vehicles Using Reinforcement Learning

Longitudinal Dynamic versus Kinematic Models for Car-Following Control Using Deep Reinforcement Learning

Bilateral Deep Reinforcement Learning Approach for Better-than-human Car Following Model

An enhanced eco-driving strategy based on reinforcement learning for connected electric vehicles: cooperative velocity and lane-changing control

Physics-informed Deep Reinforcement Learning-Based Integrated Two-Dimensional Car-Following Control Strategy for Connected Automated Vehicles

Reinforcement Learning-Based High-Speed Path Following Control for Autonomous Vehicles

Anti-Disturbance Self-Supervised Reinforcement Learning for Perturbed Car-Following System

Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning

Car‐following Strategy of Intelligent Connected Vehicle Using Extended Disturbance Observer Adjusted by Reinforcement Learning

Hybrid car following control for CAVs: Integrating linear feedback and deep reinforcement learning to stabilize mixed traffic

Safe, Efficient, and Comfortable Autonomous Driving Based on Cooperative Vehicle Infrastructure System

Deep Reinforcement Learning Car-Following Model Considering Longitudinal and Lateral Control

Distributed Connected Automated Vehicles Control under Real‐Time Aggregated Macroscopic Car Following Behavior Estimation Based on Deep Reinforcement Learning

A Combined Reinforcement Learning and Model Predictive Control for Car-Following Maneuver of Autonomous Vehicles

Connected Automated Vehicle Cooperative Control with a Deep Reinforcement Learning Approach in a Mixed Traffic Environment

Deep Adaptive Control: Deep Reinforcement Learning-Based Adaptive Vehicle Trajectory Control Algorithms for Different Risk Levels

Optimal reinforcement learning and probabilistic-risk-based path planning and following of autonomous vehicles with obstacle avoidance

A reinforcement learning based autonomous vehicle control in diverse daytime and weather scenarios