Abstract:We demonstrate the first application of deep reinforcement learning to autonomous driving. From randomly initialised parameters, our model is able to learn a policy for lane following in a handful of training episodes using a single monocular image as input. We provide a general and easy to obtain reward: the distance travelled by the vehicle without the safety driver taking control. We use a continuous, model-free deep reinforcement learning algorithm, with all exploration and optimisation performed on-vehicle. This demonstrates a new framework for autonomous driving which moves away from reliance on defined logical rules, mapping, and direct supervision. We discuss the challenges and opportunities to scale this approach to a broader range of autonomous driving tasks.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? The problem this paper attempts to solve is **how to use deep reinforcement learning (Deep Reinforcement Learning, DRL) to enable self - driving cars to learn to follow lanes in a short time**. Specifically, the authors show how to model the self - driving task as a Markov decision process (MDP) and use a continuous, model - free deep reinforcement learning algorithm (such as DDPG, Deep Deterministic Policy Gradients) to train self - driving vehicles. #### Main problems and challenges 1. **Traditional methods relying on rules and maps are difficult to scale**: - Most existing self - driving systems rely on detailed 3D geometric maps and predefined logical rules, and this method is difficult to scale to complex and variable driving scenarios. - These systems require a large amount of external infrastructure support, such as high - precision maps and complex perception modules, increasing the complexity and cost of the system. 2. **Implementing an end - to - end learning framework**: - The author hopes to develop a self - driving system that can rely solely on the understanding of the local environment without the need for detailed maps or explicit rules, similar to the way human drivers do. - Use a single forward - looking camera image as input and directly learn driving strategies from the environment through reinforcement learning, without the need for pre - labeled data or complex perception modules. 3. **Fast learning and adaptation**: - One of the goals of the paper is to let self - driving vehicles learn basic driving tasks, such as lane following, within a very short time (for example, within a few hours). - This fast - learning ability is very important for practical applications because it can reduce the time and cost of testing and verification. #### Overview of solutions - **Model the self - driving task as an MDP**: - Define the state space (S), action space (A) and reward function (R). The state space includes images from the monocular camera as well as the vehicle's speed and steering angle; the action space includes the steering angle and speed set points; the reward function is based on the distance the vehicle travels without deviating from the lane. - **Use the deep reinforcement learning algorithm (DDPG)**: - DDPG is a model - free reinforcement learning algorithm suitable for continuous action spaces and can quickly learn simple driving tasks in a simulated environment. - In the real world, using the same algorithm and hyperparameter settings, the vehicle has successfully learned the lane - following task after several training sessions. - **Experimental verification**: - Conduct preliminary verification in a simulated environment and adjust the hyperparameters to ensure the effectiveness of the algorithm. - Experiments were carried out on real roads, and the results show that the vehicle can learn the lane - following task within about 30 minutes, and all calculations were completed on the on - board computer. Through these methods, the paper demonstrates the potential of deep reinforcement learning in self - driving and provides a new direction for further research.

Learning to Drive in a Day

Learning an Efficient and Safe Policy for Highway Driving Using Supervised Learning and Reinforcement Learning.

End-to-End Deep Reinforcement Learning for Lane Keeping Assist

Deep Reinforcement Learning framework for Autonomous Driving

Learning to drive via Apprenticeship Learning and Deep Reinforcement Learning

FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing

Multi-Vehicle Mixed-Reality Reinforcement Learning for Autonomous Multi-Lane Driving

Learning to Drive Like Human Beings: A Method Based on Deep Reinforcement Learning

From Naturalistic Traffic Data to Learning-Based Driving Policy: A Sim-to-Real Study

Learning predictive representations in autonomous driving to improve deep reinforcement learning

Automated Driving Maneuvers under Interactive Environment based on Deep Reinforcement Learning

Self-Driving Car Racing: Application of Deep Reinforcement Learning

Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning

Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios

DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving

An End-to-End Curriculum Learning Approach for Autonomous Driving Scenarios

Scaling Is All You Need: Autonomous Driving with JAX-Accelerated Reinforcement Learning

Learning Robust Control Policies for End-to-End Autonomous Driving From Data-Driven Simulation

Deep reinforcement learning for autonomous driving in uncontrolled intersections of Indian roads

ReinforcementDriving: Exploring Trajectories and Navigation for Autonomous Vehicles