Abstract:Success in racing requires a unique combination of vehicle setup, understanding of the racetrack, and human expertise. Since building and testing many different vehicle configurations in the real world is prohibitively expensive, high-fidelity simulation is a critical part of racecar development. However, testing different vehicle configurations still requires expert human input in order to evaluate their performance on different racetracks. In this work, we present the first steps towards an autonomous test driver, trained using deep reinforcement learning, capable of evaluating changes in vehicle setup on racing performance while driving at the level of the best human drivers. In addition, the autonomous driver model can be tuned to exhibit more human-like behavioral patterns by incorporating imitation learning into the RL training process. This extension permits the possibility of driver-specific vehicle setup optimization.

What problem does this paper attempt to address?

### Problems the paper attempts to solve The paper aims to develop an autonomous test driver to evaluate the impact of different vehicle configurations on racing performance. Specifically, the paper trains a model through Deep Reinforcement Learning (DRL) to enable it to drive in a simulated environment and evaluate the performance of various vehicle settings, reaching or even exceeding the level of top - human drivers. #### Main problems 1. **Expensive and time - consuming real - world testing**: Testing different vehicle configurations in the real world is very expensive and time - consuming. High - fidelity simulation is a crucial part of racing car development, but it still requires expert manual input to evaluate performance on different tracks. 2. **Evaluation of vehicle limit performance**: In order to accurately evaluate the limit performance of a vehicle, the control algorithm must be able to adjust its driving style like a professional human driver to adapt to changes in vehicle handling. 3. **Personalized driving styles**: Different professional drivers have different driving styles, so sometimes it is more important to build a model that can imitate a specific driver than to pursue the fastest speed. #### Solutions The paper proposes an autonomous test - driver model based on deep reinforcement learning, which can: - Evaluate performance under different vehicle configurations and drive the vehicle to the level of top - human drivers. - By introducing imitation learning, enable the autonomous - driver model to exhibit more human - like behavior patterns, thereby achieving vehicle - configuration optimization for specific drivers. #### Specific methods - **Multi - task learning environment**: Train the agent in a multi - task environment to adapt to different vehicle configurations. - **Conditional policy**: Adjust the policy according to relevant vehicle parameters, enabling the agent to adapt to different vehicle settings. - **Reward function design**: Design complex reward functions, including progress rewards, edge penalties, tire - wear penalties, etc., to ensure that the agent not only pursues the fastest lap time but also follows track limits and limits tire wear. - **Imitation - learning extension**: By adding an imitation - loss term, make the agent's actions closer to the behaviors in human - demonstration data. Through these methods, the paper shows how to use deep reinforcement learning and imitation learning to create an efficient autonomous test driver, thereby accelerating the vehicle - development process and reducing costs.

Towards an Autonomous Test Driver: High-Performance Driver Modeling via Reinforcement Learning

An Adaptive Human Driver Model for Realistic Race Car Simulations

Personalized Driver Model and Its Application to Vehicle Testing

Autonomous Racing using a Hybrid Imitation-Reinforcement Learning Architecture

FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing

Racing Driver Modeling Based on Driving Behavior

DeepRacing: Parameterized Trajectories for Autonomous Racing

Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learning

Formula RL: Deep Reinforcement Learning for Autonomous Racing using Telemetry Data

A Simulation Benchmark for Autonomous Racing with Large-Scale Human Data

Autonomous Driving at the Handling Limit Using Residual Reinforcement Learning

Think2Drive: Efficient Reinforcement Learning by Thinking with Latent World Model for Autonomous Driving (in CARLA-v2)

End-To-End Training and Testing Gamification Framework to Learn Human Highway Driving

Think2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving (in CARLA-v2)

Racing Towards Reinforcement Learning based control of an Autonomous Formula SAE Car

Self-Driving Car Racing: Application of Deep Reinforcement Learning

Learning from Simulation, Racing in Reality

High-speed Autonomous Racing using Trajectory-aided Deep Reinforcement Learning

Comprehensive Training and Evaluation on Deep Reinforcement Learning for Automated Driving in Various Simulated Driving Maneuvers

End-to-end Methods for Autonomous Driving in Simulation

Learning Autonomous Race Driving with Action Mapping Reinforcement Learning