Abstract:Connected automated vehicles (CAVs) are broadly recognized as next-generation transformative transportation technologies having great potential to improve traffic safety, efficiency, and stability. Efficiently controlling CAVs on two-dimensional curvilinear roadways to follow preceding vehicles is denoted as the two-dimensional car-following process, which is highly critical; this process is challenging to implement owing to the complexity and varied nature of driving environments. This study proposes an innovative integrated two-dimensional control strategy for CAVs based on deep reinforcement learning (DRL), which efficiently regulates the two-dimensional car-following process of CAVs in terms of both stability-wise longitudinal control performance and accurate lateral path-tracking performance. Within the control framework, each CAV can receive the surrounding information from downstream vehicles and roadway geometry based on vehicle-to-everything (V2X) communication. To better utilize this information, we designed a physics-informed DRL state fusion approach and reward function, which efficiently embeds prior physics knowledge and borrows the merits of the equilibrium and consensus concepts from the control theory. Given the physics-informed information, the DRL-based controller outputs the integrated control instructions for both longitudinal and lateral control. For training, we constructed a roadway with a set of varying curvatures and embedded the ground-truth vehicle trajectory datasets to more effectively capture the realistic variations in the roadway geometry and driving environment. To facilitate value function approximation and enhance the policy iteration process in training, the distributed proximal policy optimization (DPPO) algorithm was applied, owing to its balanced performance. A series of simulated experiments were conducted to validate the controller’s lateral control accuracy and stability-wise oscillation dampening performance in diverse traffic scenarios, including extreme ones.

Game-theoretic Receding-Horizon Reinforcement Learning for Lateral Control of Autonomous Vehicles

Cooperative Path Following Control in Autonomous Vehicles Graphical Games: A Data-Based Off-Policy Learning Approach

Receding Horizon Reinforcement Learning Algorithm for Lateral Control of Intelligent Vehicles

A Learning-Based Flexible Autonomous Motion Control Method for UAV in Dynamic Unknown Environments

Continuous‐time receding‐horizon reinforcement learning and its application to path‐tracking control of autonomous ground vehicles

A Parameter Adaptive Trajectory Tracking and Motion Control Framework for Autonomous Vehicle

Finite-time Optimized Robust Control with Adaptive State Estimation Algorithm for Autonomous Heavy Vehicle

Robust learning-based lateral tracking control for autonomous driving with input constraints

Autonomous Driving at the Handling Limit Using Residual Reinforcement Learning

A Learning-Based Controller for Trajectory Tracking of Autonomous Vehicles in Complex and Uncertain Scenarios

Interactive Control Paradigm-Based Robust Lateral Stability Controller Design for Autonomous Automobile Path Tracking With Uncertain Disturbance: A Dynamic Game Approach.

Physics-informed Deep Reinforcement Learning-Based Integrated Two-Dimensional Car-Following Control Strategy for Connected Automated Vehicles

Optimal Lateral Path-Tracking Control of Vehicles with Partial Unknown Dynamics Via DPG-Based Reinforcement Learning Methods

Game Theory-Based Control Strategy For Trajectory Following of Four-Wheel Independently Actuated Autonomous Vehicles

Lateral Control for Autonomous Land Vehicles Via Dual Heuristic Programming

Performance-Guaranteed Adaptive Optimized Control of Intelligent Surface Vehicle Using Reinforcement Learning

Reinforcement-learning-aided Adaptive Control for Autonomous Driving with Combined Lateral and Longitudinal Dynamics

Real-Time Self-Ajusting LQR Method for Autonomous Vehicle Lateral Control

Research on trajectory tracking control of driverless cars based on game theory

Safe Reinforcement Learning for Model-Reference Trajectory Tracking of Uncertain Autonomous Vehicles with Model-Based Acceleration

Data-Driven Lateral Fault-Tolerance Control Of Autonomous Vehicle System Using Reinforcement Learning