Abstract:The path tracking control system is a crucial component for autonomous vehicles; it is challenging to realize accurate tracking control when approaching a wide range of uncertain situations and dynamic environments, particularly when such control must perform as well as, or better than, human drivers. While many methods provide state-of-the-art tracking performance, they tend to emphasize constant PID control parameters, calibrated by human experience, to improve tracking accuracy. A detailed analysis shows that PID controllers inefficiently reduce the lateral error under various conditions, such as complex trajectories and variable speed. In addition, intelligent driving vehicles are highly non-linear objects, and high-fidelity models are unavailable in most autonomous systems. As for the model-based controller (MPC or LQR), the complex modeling process may increase the computational burden. With that in mind, a self-optimizing, path tracking controller structure, based on reinforcement learning, is proposed. For the lateral control of the vehicle, a steering method based on the fusion of the reinforcement learning and traditional PID controllers is designed to adapt to various tracking scenarios. According to the pre-defined path geometry and the real-time status of the vehicle, the interactive learning mechanism, based on an RL framework (actor–critic—a symmetric network structure), can realize the online optimization of PID control parameters in order to better deal with the tracking error under complex trajectories and dynamic changes of vehicle model parameters. The adaptive performance of velocity changes was also considered in the tracking process. The proposed controlling approach was tested in different path tracking scenarios, both the driving simulator platforms and on-site vehicle experiments have verified the effects of our proposed self-optimizing controller. The results show that the approach can adaptively change the weights of PID to maintain a tracking error (simulation: within ±0.071 m; realistic vehicle: within ±0.272 m) and steering wheel vibration standard deviations (simulation: within ±0.04°; realistic vehicle: within ±80.69°); additionally, it can adapt to high-speed simulation scenarios (the maximum speed is above 100 km/h and the average speed through curves is 63–76 km/h).

Design of the Controller Based on Reinforcement Learning and Its Application on Truck Backer-Upper

Autonomous Vehicle Control Through the Dynamics and Controller Learning

Fuzzy Controller with Neuron Self-Tuning Parameters

A fuzzy logic controller designed based on CPN and its application

Hierarchical Fuzzy Control for Urban Traffic Trunk Roads and Its Neural Network Implementation

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

Chaos Control Simulation of Freeway by Using T-S Fuzzy-neural Networks Based on Rough Sets

Learning-Based Neural Dynamic Surface Predictive Control for MMC

Longitudinal deep truck: Deep longitudinal model with application to sim2real deep reinforcement learning for heavy‐duty truck control in the field

Design of Car Control Algorithm Based on Deep Reinforcement Learning

A Combined Reinforcement Learning and Model Predictive Control for Car-Following Maneuver of Autonomous Vehicles

Adaptive Learning-Based Path-Tracking Control for Unknown Vehicle Systems under Performance Optimization

Research on Modeling and Simulation of Longitudinal Control of Truck Autonomous Driving

Controller Design for Optimizing Fuel Consumption of Truck Platoon on Hilly Roads

Longitudinal Deep Truck: Deep learning and deep reinforcement learning for modeling and control of longitudinal dynamics of heavy duty trucks

Multi-objective Seamless Self-scheduling Controller Design for Heavy Commercial Vehicle Lateral Automation: An LPV/ HHH ∞∞∞ Approach

Performance-Guaranteed Adaptive Optimized Control of Intelligent Surface Vehicle Using Reinforcement Learning

Fuzzy Logic based Autonomous Parking Systems -- Part IV: A Multiple-Model Adaptive Neural-Fuzzy Controller

Fuzzy Self-adjusted and Realtime Learning Algorithm for Vehicle Lateral Control

Self-Optimizing Path Tracking Controller for Intelligent Vehicles Based on Reinforcement Learning

Modeling and Control for Mine Sweeping Plough Electro-hydraulic Servo System of Certain Mine-clearing Vehicle