Abstract:Connected automated vehicles (CAVs) are broadly recognized as next-generation transformative transportation technologies having great potential to improve traffic safety, efficiency, and stability. Efficiently controlling CAVs on two-dimensional curvilinear roadways to follow preceding vehicles is denoted as the two-dimensional car-following process, which is highly critical; this process is challenging to implement owing to the complexity and varied nature of driving environments. This study proposes an innovative integrated two-dimensional control strategy for CAVs based on deep reinforcement learning (DRL), which efficiently regulates the two-dimensional car-following process of CAVs in terms of both stability-wise longitudinal control performance and accurate lateral path-tracking performance. Within the control framework, each CAV can receive the surrounding information from downstream vehicles and roadway geometry based on vehicle-to-everything (V2X) communication. To better utilize this information, we designed a physics-informed DRL state fusion approach and reward function, which efficiently embeds prior physics knowledge and borrows the merits of the equilibrium and consensus concepts from the control theory. Given the physics-informed information, the DRL-based controller outputs the integrated control instructions for both longitudinal and lateral control. For training, we constructed a roadway with a set of varying curvatures and embedded the ground-truth vehicle trajectory datasets to more effectively capture the realistic variations in the roadway geometry and driving environment. To facilitate value function approximation and enhance the policy iteration process in training, the distributed proximal policy optimization (DPPO) algorithm was applied, owing to its balanced performance. A series of simulated experiments were conducted to validate the controller’s lateral control accuracy and stability-wise oscillation dampening performance in diverse traffic scenarios, including extreme ones.

An Efficient Deep Reinforcement Learning-based Car-following Method via Rule-constrained Data Augmentation

Path Following for Autonomous Ground Vehicle Using DDPG Algorithm: A Reinforcement Learning Approach

Deep Reinforcement Learning Car-Following Control Based on Multivehicle Motion Prediction

Towards robust car-following based on deep reinforcement learning

Improved Deep Reinforcement Learning for Car-Following Decision-Making

Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning

A Decision-Making Strategy for Car Following Based on Naturalist Driving Data via Deep Reinforcement Learning

A Deep Reinforcement Learning Based Car Following Model for Electric Vehicle

Feedback Forecasting Based Deep Deterministic Policy Gradient Algorithm for Car-Following of Autonomous Vehicle

Constrained Reinforcement Learning for Vehicle Motion Planning with Topological Reachability Analysis

Deep Reinforcement Learning Car-Following Model Considering Longitudinal and Lateral Control

Research into Autonomous Vehicles Following and Obstacle Avoidance Based on Deep Reinforcement Learning Method under Map Constraints

Physics-informed Deep Reinforcement Learning-Based Integrated Two-Dimensional Car-Following Control Strategy for Connected Automated Vehicles

Multi-agent reinforcement learning for ecological car-following control in mixed traffic

CAV-AHDV-CAV: Mitigating Traffic Oscillations for CAVs through a Novel Car-Following Structure and Reinforcement Learning

Bilateral Deep Reinforcement Learning Approach for Better-than-human Car Following Model

Optimization of Safety and Comfort in Car-following Scene Based on Reinforcement Learning

FollowGen: A Scaled Noise Conditional Diffusion Model for Car-Following Trajectory Prediction

Combining Deep Reinforcement Learning and Safety Based Control for Autonomous Driving

Path Following with Supervised Deep Reinforcement Learning

Car-Following Safe Headway Strategy with Battery-Health Conscious: A Reinforcement Learning Approach.