Abstract:Lane change is a common-yet-challenging driving behavior for automated vehicles. To improve the safety and efficiency of automated vehicles, researchers have proposed various lane-change decision models. However, most of the existing models consider lane-change behavior as a one-player decision-making problem, ignoring the essential multi-agent properties when vehicles are driving in traffic. Such models lead to deficiencies in interaction and collaboration between vehicles, which results in hazardous driving behaviors and overall traffic inefficiency. In this paper, we revisit the lane-change problem and propose a bi-level lane-change behavior planning strategy, where the upper level is a novel multi-agent deep reinforcement learning (DRL) based lane-change decision model and the lower level is a negotiation based right-of-way assignment model. We promote the collaboration performance of the upper-level lane-change decision model from three crucial aspects. First, we formulate the lane-change decision problem with a novel multi-agent reinforcement learning model, which provides a more appropriate paradigm for collaboration than the single-agent model. Second, we encode the driving intentions of surrounding vehicles into the observation space, which can empower multiple vehicles to implicitly negotiate the right-of-way in decision-making and enable the model to determine the right-of-way in a collaborative manner. Third, an ingenious reward function is designed to allow the vehicles to consider not only ego benefits but also the impact of changing lanes on traffic, which will guide the multi-agent system to learn excellent coordination performance. With the upper-level lane-change decisions, the lower-level right-of-way assignment model is used to guarantee the safety of lane-change behaviors. The experiments show that the proposed approaches can lead to safe, efficient, and harmonious lane-change behaviors, which boosts the collaboration between vehicles and in turn improves the safety and efficiency of the overall traffic. Moreover, the proposed approaches promote the microscopic synchronization of vehicles, which can lead to the macroscopic synchronization of traffic flow.

Reinforcement Learning for Freeway Lane-Change Regulation via Connected Vehicles

A Reinforcement Learning Approach to Smart Lane Changes of Self-driving Cars

Developing Smart Lane-changing Strategies for CAVs on Freeways based on MOBIL and Reinforcement Learning

A Q-learning Foresighted Approach to Ego-efficient Lane Changes of Connected and Automated Vehicles on Freeways.

Modeling Heterogeneous Traffic at Ramp with Autonomous Vehicles Based on Cellular Automata and Multi-Agent System

LCS-TF: Multi-Agent Deep Reinforcement Learning-Based Intelligent Lane-Change System for Improving Traffic Flow

Decentralized Cooperative Lane Changing at Freeway Weaving Areas Using Multi-Agent Deep Reinforcement Learning

Multi-Agent Deep Reinforcement Learning for Multi-Lane Freeways Differential Variable Speed Limit Control in Mixed Traffic Environment

A Machine Learning Method for Dynamic Traffic Control and Guidance on Freeway Networks

Multi-agent reinforcement learning for cooperative lane changing of connected and autonomous vehicles in mixed traffic

Multi-Agent DRL-Based Lane Change with Right-of-Way Collaboration Awareness

Mandatory Lane-Changing Decision-Making in Dense Traffic for Autonomous Vehicles Based on Deep Reinforcement Learning

Multi-Task Lane-Free Driving Strategy for Connected and Automated Vehicles: A Multi-Agent Deep Reinforcement Learning Approach

A multi-agent reinforcement learning-based longitudinal and lateral control of CAVs to improve traffic efficiency in a mandatory lane change scenario

An Evolutionary Learning Framework of Lane-Changing Control for Autonomous Vehicles at Freeway Off-Ramps

Reinforcement Learning from Human Feedback for Lane Changing of Autonomous Vehicles in Mixed Traffic

An Improved Hierarchical Deep Reinforcement Learning Algorithm for Multi-Intelligent Vehicle Lane Change

Chance-Aware Lane Change with High-Level Model Predictive Control Through Curriculum Reinforcement Learning

A New Solution for Freeway Congestion: Cooperative Speed Limit Control Using Distributed Reinforcement Learning

Modeling Coupled Driving Behavior During Lane Change: A Multi-Agent Transformer Reinforcement Learning Approach

A Lane Changing Model of Automatic Driving Vehicle Based on Reinforcement Learning.