Autonomous vehicle decision and control through reinforcement learning with traffic flow randomization

Yuan Lin,Antai Xie,Xiao Liu

DOI: https://doi.org/10.3390/machines12040264

2024-04-19

Abstract:Most of the current studies on autonomous vehicle decision-making and control tasks based on reinforcement learning are conducted in simulated environments. The training and testing of these studies are carried out under rule-based microscopic traffic flow, with little consideration of migrating them to real or near-real environments to test their performance. It may lead to a degradation in performance when the trained model is tested in more realistic traffic scenes. In this study, we propose a method to randomize the driving style and behavior of surrounding vehicles by randomizing certain parameters of the car-following model and the lane-changing model of rule-based microscopic traffic flow in SUMO. We trained policies with deep reinforcement learning algorithms under the domain randomized rule-based microscopic traffic flow in freeway and merging scenes, and then tested them separately in rule-based microscopic traffic flow and high-fidelity microscopic traffic flow. Results indicate that the policy trained under domain randomization traffic flow has significantly better success rate and calculative reward compared to the models trained under other microscopic traffic flows.

Systems and Control,Machine Learning,Robotics

What problem does this paper attempt to address?

The paper aims to address the performance degradation issue faced by autonomous vehicles during the decision-making and control process based on reinforcement learning when transitioning from a simulation environment to a real or near-real traffic environment. Specifically:

1. **Limitations of Existing Research**: Most current research on autonomous driving decision-making and control based on reinforcement learning is conducted in regulated microscopic traffic flow simulation experiments. These studies usually overlook the performance of transferring the trained models to more realistic traffic scenarios.

2. **Proposed Method**: The paper proposes a method that randomizes certain parameters in the regulated microscopic traffic flow (such as car-following models and lane-changing models) to make the behavior of surrounding vehicles more uncertain. This method is called domain randomization.

3. **Experimental Validation**: The authors use deep reinforcement learning algorithms to train policies in the randomized regulated microscopic traffic flow and test these policies in both regulated and high-fidelity microscopic traffic flows. The results show that the policies trained under randomized traffic flow exhibit better success rates and reward values across different types of traffic flows.

In summary, the core issue of the paper is to improve the adaptability and robustness of reinforcement learning-based autonomous driving systems in different traffic environments.