Abstract:Lane change is a common-yet-challenging driving behavior for automated vehicles. To improve the safety and efficiency of automated vehicles, researchers have proposed various lane-change decision models. However, most of the existing models consider lane-change behavior as a one-player decision-making problem, ignoring the essential multi-agent properties when vehicles are driving in traffic. Such models lead to deficiencies in interaction and collaboration between vehicles, which results in hazardous driving behaviors and overall traffic inefficiency. In this paper, we revisit the lane-change problem and propose a bi-level lane-change behavior planning strategy, where the upper level is a novel multi-agent deep reinforcement learning (DRL) based lane-change decision model and the lower level is a negotiation based right-of-way assignment model. We promote the collaboration performance of the upper-level lane-change decision model from three crucial aspects. First, we formulate the lane-change decision problem with a novel multi-agent reinforcement learning model, which provides a more appropriate paradigm for collaboration than the single-agent model. Second, we encode the driving intentions of surrounding vehicles into the observation space, which can empower multiple vehicles to implicitly negotiate the right-of-way in decision-making and enable the model to determine the right-of-way in a collaborative manner. Third, an ingenious reward function is designed to allow the vehicles to consider not only ego benefits but also the impact of changing lanes on traffic, which will guide the multi-agent system to learn excellent coordination performance. With the upper-level lane-change decisions, the lower-level right-of-way assignment model is used to guarantee the safety of lane-change behaviors. The experiments show that the proposed approaches can lead to safe, efficient, and harmonious lane-change behaviors, which boosts the collaboration between vehicles and in turn improves the safety and efficiency of the overall traffic. Moreover, the proposed approaches promote the microscopic synchronization of vehicles, which can lead to the macroscopic synchronization of traffic flow.

A multi-agent reinforcement learning-based longitudinal and lateral control of CAVs to improve traffic efficiency in a mandatory lane change scenario

Developing Smart Lane-changing Strategies for CAVs on Freeways based on MOBIL and Reinforcement Learning

A Reinforcement Learning Approach to Smart Lane Changes of Self-driving Cars

A Q-learning Foresighted Approach to Ego-efficient Lane Changes of Connected and Automated Vehicles on Freeways.

Mandatory Lane-Changing Decision-Making in Dense Traffic for Autonomous Vehicles Based on Deep Reinforcement Learning

Multi-Agent DRL-Based Lane Change with Right-of-Way Collaboration Awareness

Multi-agent reinforcement learning for cooperative lane changing of connected and autonomous vehicles in mixed traffic

Multi-Task Lane-Free Driving Strategy for Connected and Automated Vehicles: A Multi-Agent Deep Reinforcement Learning Approach

An Improved Hierarchical Deep Reinforcement Learning Algorithm for Multi-Intelligent Vehicle Lane Change

Simulation and Evaluation of Speed and Lane-Changing Advisory of CAVS at Work Zones in Heterogeneous Traffic Flow

Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion

Cooperative Lane-Change Control Method for Freeways Considering Dynamic Intelligent Connected Dedicated Lanes

A Novel Dynamic Lane-Changing Trajectory Planning Model for Automated Vehicles Based on Reinforcement Learning

A heuristic-action-involved safe lane-change of autonomous vehicles with multiple constraints on road centerline and speed under highway environment

Cooperative lane-changing in mixed traffic: a deep reinforcement learning approach

A Cooperative Control Framework for CAV Lane Change in a Mixed Traffic Environment

A Deep Reinforcement Learning-Based Approach for Autonomous Lane-Changing Velocity Control in Mixed Flow of Vehicle Group Level

An Evolutionary Learning Framework of Lane-Changing Control for Autonomous Vehicles at Freeway Off-Ramps

LCS-TF: Multi-Agent Deep Reinforcement Learning-Based Intelligent Lane-Change System for Improving Traffic Flow

Iterative learning control for lane-changing trajectories upstream off-ramp bottlenecks and safety evaluation

A Lane Changing Model of Automatic Driving Vehicle Based on Reinforcement Learning.