Abstract:Executing drift maneuvers during high-speed cornering presents significant challenges for autonomous vehicles, yet offers the potential to minimize turning time and enhance driving dynamics. While reinforcement learning (RL) has shown promising results in simulated environments, discrepancies between simulations and real-world conditions have limited its practical deployment. This study introduces an innovative control framework that integrates trajectory optimization with drift maneuvers, aiming to improve the algorithm's adaptability for real-vehicle implementation. We leveraged Bezier-based pre-trajectory optimization to enhance rewards and optimize the controller through Twin Delayed Deep Deterministic Policy Gradient (TD3) in a simulated environment. For real-world deployment, we implement a hybrid RL-MPC fusion mechanism, , where TD3-derived maneuvers serve as primary inputs for a Model Predictive Controller (MPC). This integration enables precise real-time tracking of the optimal trajectory, with MPC providing corrective inputs to bridge the gap between simulation and reality. The efficacy of this method is validated through real-vehicle tests on consumer-grade electric vehicles, focusing on drift U-turns and drift right-angle turns. The control outcomes of these real-vehicle tests are thoroughly documented in the paper, supported by supplementary video evidence (<a class="link-external link-https" href="https://youtu.be/5wp67FcpfL8" rel="external noopener nofollow">this https URL</a>). Notably, this study is the first to deploy and apply an RL-based transient drift cornering algorithm on consumer-grade electric vehicles.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: the challenges faced by autonomous electric vehicles in performing drift maneuvers during high - speed cornering. Specifically, the authors aim to improve the adaptability of algorithms for actual vehicle deployment and reduce the gap from the simulated environment to the real world by combining trajectory optimization and drift control strategies. ### Specific description of the problem 1. **Drift control in high - speed cornering**: - Autonomous vehicles face significant challenges when performing drift maneuvers during high - speed cornering, but such operations can minimize the turning time and enhance the driving dynamics. - Reinforcement Learning (RL) has shown promising results in the simulated environment, but in practical applications, its effectiveness is limited due to the differences between simulated and real - world conditions. 2. **The gap from simulation to reality**: - Although RL performs well in the simulated environment, the complexity and unpredictability in the actual vehicle traffic environment make these methods difficult to be directly applied to the real world. - The dynamic models and road conditions in the simulated environment often cannot fully capture the complexity and variability of real vehicles and scenarios, resulting in a decline in performance and safety. 3. **Limitations of existing research**: - Existing research mainly focuses on the simulated environment. Although some achievements have been made, its application in actual vehicles still needs further verification. - Most of the existing drift control strategies rely on a hierarchical trajectory planning - tracking architecture and are difficult to optimize the minimum turning time on complex curves. ### Solution To solve the above problems, the authors propose an innovative control framework that combines data - driven and model - based tracking strategies: 1. **Combination of TD3 controller and pre - trajectory optimization**: - A TD3 (Twin Delayed Deep Deterministic Policy Gradient) - based controller is proposed, which, combined with pre - trajectory optimization, can perform transient drift cornering on complex curves and optimize the turning time. 2. **RL - MPC fusion mechanism**: - A hybrid control framework that combines RL and MPC (Model Predictive Controller) is designed, which improves the adaptability of actual vehicle deployment and solves the gap from simulation to reality. - The actions generated by TD3 are used as the main input, and MPC provides corrective inputs to ensure accurate real - time trajectory tracking. 3. **Actual vehicle verification**: - The effectiveness of this control framework has been verified on consumer - level electric vehicles. U - turns and right - angle turns have been successfully executed, demonstrating the application ability of this controller in unpredictable environments. ### Main contributions 1. **A TD3 - based controller is proposed**, which, combined with pre - trajectory optimization, can perform transient drift cornering and optimize the turning time on complex curves. 2. **An RL - MPC fusion control framework is designed**, which improves the adaptability of actual vehicle deployment and solves the gap from simulation to reality. 3. **For the first time, an RL - based transient drift cornering algorithm has been deployed and applied on consumer - level electric vehicles**, which significantly improves the performance of autonomous racing cars and enhances the handling ability of consumer - level vehicles in high - speed cornering. Through these improvements, the authors not only solve the control problems of autonomous electric vehicles during high - speed cornering but also provide a feasible solution for the practical application of reinforcement learning in unpredictable environments.

High-Speed Cornering Control and Real-Vehicle Deployment for Autonomous Electric Vehicles

An Aggressive Cornering Framework for Autonomous Vehicles Combining Trajectory Planning and Drift Control

An MPC-based Controller Framework for Agile Maneuvering of Autonomous Vehicles.

Learning-Based MPC Controller for Drift Control of Autonomous Vehicles

Combined Fast Control of Drifting State and Trajectory Tracking for Autonomous Vehicles Based on MPC Controller

Learning-Based Hierarchical Model Predictive Control for Drift Vehicles

A Fast Cornering Method Based on Model Predictive Control

Drift control for cornering maneuver of autonomous vehicles

Curriculum reinforcement learning-based drifting along a general path for autonomous vehicles

Autonomous Drift Cornering With Mixed Open-Loop And Closed-Loop Control

A novel model predictive controller for the drifting vehicle to track a circular trajectory

A Hierarchical Control Framework for Drift Maneuvering of Autonomous Vehicles

Deep reinforcement learning-based drift parking control of automated vehicles

Longitudinal control of automated vehicles: A novel approach by integrating deep reinforcement learning with intelligent driver model

Coupled Lateral and Longitudinal Control for Trajectory Tracking, Lateral Stability, and Rollover Prevention of High-Speed Automated Vehicles Using Minimum-Time Model Predictive Control

Real-Time Drift-Driving Control for an Autonomous Vehicle: Learning from Nonlinear Model Predictive Control via a Deep Neural Network

Reinforcement Learning-Based High-Speed Path Following Control for Autonomous Vehicles

Learning to Race in Extreme Turning Scene with Active Exploration and Gaussian Process Regression-based MPC

Data-Driven Trajectory Tracking Control Algorithm Design for Fast Migration to Different Autonomous Vehicles

A Nonlinear Model Predictive Control for Automated Drifting with a Standard Passenger Vehicle

Autonomous Driving at the Handling Limit Using Residual Reinforcement Learning