Abstract:Connected automated vehicles (CAVs) are broadly recognized as next-generation transformative transportation technologies having great potential to improve traffic safety, efficiency, and stability. Efficiently controlling CAVs on two-dimensional curvilinear roadways to follow preceding vehicles is denoted as the two-dimensional car-following process, which is highly critical; this process is challenging to implement owing to the complexity and varied nature of driving environments. This study proposes an innovative integrated two-dimensional control strategy for CAVs based on deep reinforcement learning (DRL), which efficiently regulates the two-dimensional car-following process of CAVs in terms of both stability-wise longitudinal control performance and accurate lateral path-tracking performance. Within the control framework, each CAV can receive the surrounding information from downstream vehicles and roadway geometry based on vehicle-to-everything (V2X) communication. To better utilize this information, we designed a physics-informed DRL state fusion approach and reward function, which efficiently embeds prior physics knowledge and borrows the merits of the equilibrium and consensus concepts from the control theory. Given the physics-informed information, the DRL-based controller outputs the integrated control instructions for both longitudinal and lateral control. For training, we constructed a roadway with a set of varying curvatures and embedded the ground-truth vehicle trajectory datasets to more effectively capture the realistic variations in the roadway geometry and driving environment. To facilitate value function approximation and enhance the policy iteration process in training, the distributed proximal policy optimization (DPPO) algorithm was applied, owing to its balanced performance. A series of simulated experiments were conducted to validate the controller’s lateral control accuracy and stability-wise oscillation dampening performance in diverse traffic scenarios, including extreme ones.

Composite Deep Learning Control for Autonomous Bicycles by Using Deep Deterministic Policy Gradient

Control of Unmanned Bicycle Based on Tensor Product Model Transformation and Hammersley Sampling Method

Dynamical Motor Control Learned with Deep Deterministic Policy Gradient

Cooperative Path Following Control in Autonomous Vehicles Graphical Games: A Data-Based Off-Policy Learning Approach

Deep Reinforcement Learning-Based Control of Bicycle Robots on Rough Terrain

Path Following for Autonomous Ground Vehicle Using DDPG Algorithm: A Reinforcement Learning Approach

Robust Motion Control for UAV in Dynamic Uncertain Environments Using Deep Reinforcement Learning.

Distributed Drive Autonomous Vehicle Trajectory Tracking Control Based on Multi-Agent Deep Reinforcement Learning

Dynamic modeling and balance control of human/bicycle systems

Path Tracking Control of Autonomous Ground Vehicles Via Model Predictive Control and Deep Deterministic Policy Gradient Algorithm

Learning-Based Trajectory Tracking and Balance Control for Bicycle Robots with a Pendulum: A Gaussian Process Approach

Path-Tracking Control Strategy of Unmanned Vehicle Based on DDPG Algorithm

Path tracking control based on Deep reinforcement learning in Autonomous driving

Reinforcement-learning-aided Adaptive Control for Autonomous Driving with Combined Lateral and Longitudinal Dynamics

Model-Based Ddpg for Motor Control

Steering Control and Stability Analysis for an Autonomous Bicycle through Reduced Dynamics with Servo-Constraints

Optimal Lateral Path-Tracking Control of Vehicles with Partial Unknown Dynamics Via DPG-Based Reinforcement Learning Methods

Autonomous Driving Control Using the DDPG and RDPG Algorithms

Physics-informed Deep Reinforcement Learning-Based Integrated Two-Dimensional Car-Following Control Strategy for Connected Automated Vehicles

A Novel Vehicle Platoon Following Controller Based on Deep Deterministic Policy Gradient Algorithms

A Distributed Deep Koopman Learning Algorithm for Control