Abstract:The path tracking control system is a crucial component for autonomous vehicles; it is challenging to realize accurate tracking control when approaching a wide range of uncertain situations and dynamic environments, particularly when such control must perform as well as, or better than, human drivers. While many methods provide state-of-the-art tracking performance, they tend to emphasize constant PID control parameters, calibrated by human experience, to improve tracking accuracy. A detailed analysis shows that PID controllers inefficiently reduce the lateral error under various conditions, such as complex trajectories and variable speed. In addition, intelligent driving vehicles are highly non-linear objects, and high-fidelity models are unavailable in most autonomous systems. As for the model-based controller (MPC or LQR), the complex modeling process may increase the computational burden. With that in mind, a self-optimizing, path tracking controller structure, based on reinforcement learning, is proposed. For the lateral control of the vehicle, a steering method based on the fusion of the reinforcement learning and traditional PID controllers is designed to adapt to various tracking scenarios. According to the pre-defined path geometry and the real-time status of the vehicle, the interactive learning mechanism, based on an RL framework (actor–critic—a symmetric network structure), can realize the online optimization of PID control parameters in order to better deal with the tracking error under complex trajectories and dynamic changes of vehicle model parameters. The adaptive performance of velocity changes was also considered in the tracking process. The proposed controlling approach was tested in different path tracking scenarios, both the driving simulator platforms and on-site vehicle experiments have verified the effects of our proposed self-optimizing controller. The results show that the approach can adaptively change the weights of PID to maintain a tracking error (simulation: within ±0.071 m; realistic vehicle: within ±0.272 m) and steering wheel vibration standard deviations (simulation: within ±0.04°; realistic vehicle: within ±80.69°); additionally, it can adapt to high-speed simulation scenarios (the maximum speed is above 100 km/h and the average speed through curves is 63–76 km/h).

Ultra-Fast Tuning of Neural Network Controllers with Application in Path Tracking of Autonomous Vehicle

A Learning-Based Nonlinear Model Predictive Control Approach for Autonomous Driving

Control of Nonaffine Nonlinear Discrete-Time Systems Using Reinforcement-Learning-Based Linearly Parameterized Neural Networks

Neural-network-based Learning Control for the High-Speed Path Tracking of Unmanned Ground Vehicles

Learning-Based Neural Dynamic Surface Predictive Control for MMC

Reinforcement Learning-Based High-Speed Path Following Control for Autonomous Vehicles

Longitudinal control of automated vehicles: A novel approach by integrating deep reinforcement learning with intelligent driver model

Path Tracking for Car-like Robots Based on Neural Networks with NMPC As Learning Samples

Online Reinforcement Learning Neural Network Controller Design for Nanomanipulation

Self-Optimizing Path Tracking Controller for Intelligent Vehicles Based on Reinforcement Learning

Path tracking control based on Deep reinforcement learning in Autonomous driving

Real-Time Drift-Driving Control for an Autonomous Vehicle: Learning from Nonlinear Model Predictive Control via a Deep Neural Network

Design and Experimental Validation of Deep Reinforcement Learning-Based Fast Trajectory Planning and Control for Mobile Robot in Unknown Environment

Performance-Guaranteed Adaptive Optimized Control of Intelligent Surface Vehicle Using Reinforcement Learning

Deviation Sequence Neural Network Control for Path Tracking of Autonomous Vehicles

MNNMs Integrated Control for UAV Autonomous Tracking Randomly Moving Target Based on Learning Method

Model-Based Control with Sparse Neural Dynamics

Improvement of a Learning-Based Tuning Approach for Trajectory Tracking Controller

Deep Reinforcement Learning-Based End-to-End Control for UAV Dynamic Target Tracking

Overview of the Application of Neural Networks in the Motion Control of Unmanned Vehicles

Adaptive NN event-triggered control for path following of underactuated vessels with finite-time convergence