Abstract:Intelligent decisions for autonomous lane-changing in vehicles have consistently been a focal point of research in the industry. Traditional lane-changing algorithms, which rely on predefined rules, are ill-suited for the complexities and variabilities of real-world road conditions. In this study, we propose an algorithm that leverages the deep deterministic policy gradient (DDPG) reinforcement learning, integrated with a long short-term memory (LSTM) trajectory prediction model, termed as LSTM-DDPG. In the proposed LSTM-DDPG model, the LSTM state module transforms the observed values from the observation module into a state representation, which then serves as a direct input to the DDPG actor network. Meanwhile, the LSTM prediction module translates the historical trajectory coordinates of nearby vehicles into a word-embedding vector via a fully connected layer, thus providing predicted trajectory information for surrounding vehicles. This integrated LSTM approach considers the potential influence of nearby vehicles on the lane-changing decisions of the subject vehicle. Furthermore, our study emphasizes the safety, efficiency, and comfort of the lane-changing process. Accordingly, we designed a reward and penalty function for the LSTM-DDPG algorithm and determined the optimal network structure parameters. The algorithm was then tested on a simulation platform built with MATLAB/Simulink. Our findings indicate that the LSTM-DDPG model offers a more realistic representation of traffic scenarios involving vehicle interactions. When compared to the traditional DDPG algorithm, the LSTM-DDPG achieved a 7.4% increase in average single-step rewards after normalization, underscoring its superior performance in enhancing lane-changing safety and efficiency. This research provides new ideas for advanced lane-changing decisions in autonomous vehicles.

Inverse Reinforcement Learning Based: Segmented Lane-Change Trajectory Planning with Consideration of Interactive Driving Intention

A Reinforcement Learning Approach to Smart Lane Changes of Self-driving Cars

A Dynamic Trajectory Planning Method for Lane-Changing Maneuver of Connected and Automated Vehicles

A Q-learning Foresighted Approach to Ego-efficient Lane Changes of Connected and Automated Vehicles on Freeways.

Developing Smart Lane-changing Strategies for CAVs on Freeways based on MOBIL and Reinforcement Learning

Learning from Demonstration: Situation-Adaptive Lane Change Trajectory Planning for Automated Highway Driving

A lane-changing trajectory re-planning method considering conflicting traffic scenarios

Graph-based Scenario-Adaptive Lane-Changing Trajectory Planning for Autonomous Driving

A Novel Dynamic Lane-Changing Trajectory Planning Model for Automated Vehicles Based on Reinforcement Learning

Combining Decision Making and Trajectory Planning for Lane Changing Using Deep Reinforcement Learning

Reinforcement Learning from Human Feedback for Lane Changing of Autonomous Vehicles in Mixed Traffic

Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving

Deep Reinforcement Learning Lane-Changing Decision Algorithm for Intelligent Vehicles Combining LSTM Trajectory Prediction

A Safe and Efficient Lane Change Decision-Making Strategy of Autonomous Driving Based on Deep Reinforcement Learning

A Human-Like Free-Lane-Change Trajectory Planning and Control Method With Data-Based Behavior Decision

Lane-Change Intention Inference Based on RNN for Autonomous Driving on Highways

A Human-like Trajectory Planning Method by Learning from Naturalistic Driving Data

An Improved Hierarchical Deep Reinforcement Learning Algorithm for Multi-Intelligent Vehicle Lane Change

Driving in Real Life with Inverse Reinforcement Learning

Learning from Naturalistic Driving Data for Human-like Autonomous Highway Driving