Abstract:Lane change is a common-yet-challenging driving behavior for automated vehicles. To improve the safety and efficiency of automated vehicles, researchers have proposed various lane-change decision models. However, most of the existing models consider lane-change behavior as a one-player decision-making problem, ignoring the essential multi-agent properties when vehicles are driving in traffic. Such models lead to deficiencies in interaction and collaboration between vehicles, which results in hazardous driving behaviors and overall traffic inefficiency. In this paper, we revisit the lane-change problem and propose a bi-level lane-change behavior planning strategy, where the upper level is a novel multi-agent deep reinforcement learning (DRL) based lane-change decision model and the lower level is a negotiation based right-of-way assignment model. We promote the collaboration performance of the upper-level lane-change decision model from three crucial aspects. First, we formulate the lane-change decision problem with a novel multi-agent reinforcement learning model, which provides a more appropriate paradigm for collaboration than the single-agent model. Second, we encode the driving intentions of surrounding vehicles into the observation space, which can empower multiple vehicles to implicitly negotiate the right-of-way in decision-making and enable the model to determine the right-of-way in a collaborative manner. Third, an ingenious reward function is designed to allow the vehicles to consider not only ego benefits but also the impact of changing lanes on traffic, which will guide the multi-agent system to learn excellent coordination performance. With the upper-level lane-change decisions, the lower-level right-of-way assignment model is used to guarantee the safety of lane-change behaviors. The experiments show that the proposed approaches can lead to safe, efficient, and harmonious lane-change behaviors, which boosts the collaboration between vehicles and in turn improves the safety and efficiency of the overall traffic. Moreover, the proposed approaches promote the microscopic synchronization of vehicles, which can lead to the macroscopic synchronization of traffic flow.

Efficient Lane-changing Behavior Planning Via Reinforcement Learning with Imitation Learning Initialization.

A Reinforcement Learning Approach to Smart Lane Changes of Self-driving Cars

Developing Smart Lane-changing Strategies for CAVs on Freeways based on MOBIL and Reinforcement Learning

Automated Lane Change Strategy using Proximal Policy Optimization-based Deep Reinforcement Learning

Inverse Reinforcement Learning Based: Segmented Lane-Change Trajectory Planning with Consideration of Interactive Driving Intention

Reinforcement Learning from Human Feedback for Lane Changing of Autonomous Vehicles in Mixed Traffic

A Novel Dynamic Lane-Changing Trajectory Planning Model for Automated Vehicles Based on Reinforcement Learning

A Safe and Efficient Lane Change Decision-Making Strategy of Autonomous Driving Based on Deep Reinforcement Learning

A Lane Changing Model of Automatic Driving Vehicle Based on Reinforcement Learning.

Multi-Agent DRL-Based Lane Change with Right-of-Way Collaboration Awareness

Safe Reinforcement Learning for Lane-Changing with Comprehensive Analysis of Safety Detection

Combining Decision Making and Trajectory Planning for Lane Changing Using Deep Reinforcement Learning

Continuous Control for Automated Lane Change Behavior Based on Deep Deterministic Policy Gradient Algorithm

Automated Driving Maneuvers under Interactive Environment based on Deep Reinforcement Learning

A Deep Reinforcement Learning-Based Approach for Autonomous Lane-Changing Velocity Control in Mixed Flow of Vehicle Group Level

Safe reinforcement learning for autonomous vehicles to make lane-change decisions: Constraint based on Incomplete Information Game Theory

An Improved Hierarchical Deep Reinforcement Learning Algorithm for Multi-Intelligent Vehicle Lane Change

Human Knowledge Enhanced Reinforcement Learning for Mandatory Lane-Change of Autonomous Vehicles in Congested Traffic

Mix Q-learning for Lane Changing: A Collaborative Decision-Making Method in Multi-Agent Deep Reinforcement Learning

Chance-Aware Lane Change with High-Level Model Predictive Control Through Curriculum Reinforcement Learning

Mandatory Lane-Changing Decision-Making in Dense Traffic for Autonomous Vehicles Based on Deep Reinforcement Learning