Abstract:Connected automated vehicles (CAVs) are broadly recognized as next-generation transformative transportation technologies having great potential to improve traffic safety, efficiency, and stability. Efficiently controlling CAVs on two-dimensional curvilinear roadways to follow preceding vehicles is denoted as the two-dimensional car-following process, which is highly critical; this process is challenging to implement owing to the complexity and varied nature of driving environments. This study proposes an innovative integrated two-dimensional control strategy for CAVs based on deep reinforcement learning (DRL), which efficiently regulates the two-dimensional car-following process of CAVs in terms of both stability-wise longitudinal control performance and accurate lateral path-tracking performance. Within the control framework, each CAV can receive the surrounding information from downstream vehicles and roadway geometry based on vehicle-to-everything (V2X) communication. To better utilize this information, we designed a physics-informed DRL state fusion approach and reward function, which efficiently embeds prior physics knowledge and borrows the merits of the equilibrium and consensus concepts from the control theory. Given the physics-informed information, the DRL-based controller outputs the integrated control instructions for both longitudinal and lateral control. For training, we constructed a roadway with a set of varying curvatures and embedded the ground-truth vehicle trajectory datasets to more effectively capture the realistic variations in the roadway geometry and driving environment. To facilitate value function approximation and enhance the policy iteration process in training, the distributed proximal policy optimization (DPPO) algorithm was applied, owing to its balanced performance. A series of simulated experiments were conducted to validate the controller’s lateral control accuracy and stability-wise oscillation dampening performance in diverse traffic scenarios, including extreme ones.

A deep reinforcement learning-based active suspension control algorithm considering deterministic experience tracing for autonomous vehicle

Research on Deep Reinforcement Learning Control Algorithm for Active Suspension Considering Uncertain Time Delay

Enhancing Active Disturbance Rejection Design Via Deep Reinforcement Learning and Its Application to Autonomous Vehicle

Semi-Active Suspension Control Based on Deep Reinforcement Learning

Reinforcement-learning-aided Adaptive Control for Autonomous Driving with Combined Lateral and Longitudinal Dynamics

A Learning-Based Flexible Autonomous Motion Control Method for UAV in Dynamic Unknown Environments

Physics-Guided Reinforcement Learning System for Realistic Vehicle Active Suspension Control

Path tracking control based on Deep reinforcement learning in Autonomous driving

Obstacle Avoidance for Self-Driving Vehicle with Reinforcement Learning

Longitudinal control of automated vehicles: A novel approach by integrating deep reinforcement learning with intelligent driver model

Combining Deep Reinforcement Learning and Safety Based Control for Autonomous Driving

DDPG Based Self-Learning Active and Model-Constrained Semi-Active Suspension Control

Physics-informed Deep Reinforcement Learning-Based Integrated Two-Dimensional Car-Following Control Strategy for Connected Automated Vehicles

Tracking Control for Autonomous Four-Wheel Independently Driven Vehicle Based on Deep Reinforcement Learning

A Deep Reinforcement Learning Algorithm with Expert Demonstrations and Supervised Loss and Its Application in Autonomous Driving

Stabilization of vertical motion of a vehicle on bumpy terrain using deep reinforcement learning

Autonomous Driving Control Using the DDPG and RDPG Algorithms

Deep Reinforcement Learning Car-Following Control Based on Multivehicle Motion Prediction

Autonomous Driving at the Handling Limit Using Residual Reinforcement Learning

A Deep Reinforcement Learning-Based Controller for Magnetorheological-Damped Vehicle Suspension

Stabilization Approaches for Reinforcement Learning-Based End-to-End Autonomous Driving