Abstract:Autonomous vehicles need to solve the road keeping problem and the existing solutions based on reinforcement learning are mainly implemented in the simulators. The key of transferring the well-trained models to the real world is bridging the gaps between the simulator scenarios and the real scenarios. In this paper, we propose a method called ReinforcementDriving which explores navigation skills and trajectories from simulator for full-sized road keeping. Based on the real scenario, a driving simulator is firstly established to train an intelligent driving agent. The well-trained ReinforcementDriving agent is evaluated in a real-world scenario. We compare our work with human driving, optimal control-based tracking methods and other reinforcement learning-based lane following methods. The results demonstrate that the ReinforcementDriving system can effectively achieve lane keeping in a realistic scenario with satisfactory running time and lateral accuracy.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to address the problem of road-keeping for autonomous vehicles using reinforcement learning techniques. Specifically: 1. **Combination of Reinforcement Learning and Simulators**: - Currently, most reinforcement learning-based autonomous driving applications are implemented in simulators. However, these methods are difficult to apply directly to the real world due to the gap between simulated and real environments. - The paper proposes a method called ReinforcementDriving, which first trains intelligent driving agents in a simulated environment and eventually deploys them in the real world for evaluation. 2. **Bridging the Gap Between Simulators and the Real World**: - GPS information is used to create a simulated environment that describes the road contours in real scenarios. This approach effectively maps the vehicle states in the simulated environment to the real world. - A lightweight network model is proposed to facilitate deployment and improve the real-time performance of the entire system. 3. **Solution to Continuous Control Problems**: - Autonomous driving is a continuous control problem, while some existing reinforcement learning algorithms are only suitable for discrete problems. The paper employs the DDPG (Deep Deterministic Policy Gradient) algorithm to address this issue and proposes a low-dimensional state description method and a lightweight network to ensure the system's real-time performance. 4. **Normalization Method for Executing Actions**: - A normalization method for executing actions and an exploration method with noise compensation are proposed to enhance the system's robustness and narrow the gap between the real world and the simulator. Through these methods, the paper aims to provide an effective reference solution for transferring models trained in simulators to real vehicles and demonstrates the effectiveness and real-time performance of its method through experimental results on full-size drive-by-wire vehicles.

ReinforcementDriving: Exploring Trajectories and Navigation for Autonomous Vehicles

Reinforcement-Tracking: An Effective Trajectory Tracking and Navigation Method for Autonomous Urban Driving

Learning an Efficient and Safe Policy for Highway Driving Using Supervised Learning and Reinforcement Learning.

Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer

Deep Reinforcement Learning for Autonomous Driving by Transferring Visual Features.

Learning to Drive Like Human Beings: A Method Based on Deep Reinforcement Learning

Safe Trajectory Planning Using Reinforcement Learning for Self Driving

Reinforcement Learning Based Trajectory Planning for Autonomous Vehicles

Intelligent Vehicle Decision-Making and Trajectory Planning Method Based on Deep Reinforcement Learning in the Frenet Space

Human-Guided Reinforcement Learning With Sim-to-Real Transfer for Autonomous Navigation

Road Tracking Using Deep Reinforcement Learning for Self-driving Car Applications

Human-like Highway Trajectory Modeling Based on Inverse Reinforcement Learning.

Virtual to Real Reinforcement Learning for Autonomous Driving

Autonomous Driving at the Handling Limit Using Residual Reinforcement Learning

Automated Driving Maneuvers under Interactive Environment based on Deep Reinforcement Learning

A Platform-Agnostic Deep Reinforcement Learning Framework for Effective Sim2Real Transfer towards Autonomous Driving

Joint Optimization of Sensing, Decision-making and Motion-controlling for Autonomous Vehicles: A Deep Reinforcement Learning Approach

Autonomous Highway Driving using Deep Reinforcement Learning

Reinforcement Learning from Human Feedback for Lane Changing of Autonomous Vehicles in Mixed Traffic

Path tracking control based on Deep reinforcement learning in Autonomous driving

Reinforcement Learning-Based High-Speed Path Following Control for Autonomous Vehicles