Abstract:Lane change is a common-yet-challenging driving behavior for automated vehicles. To improve the safety and efficiency of automated vehicles, researchers have proposed various lane-change decision models. However, most of the existing models consider lane-change behavior as a one-player decision-making problem, ignoring the essential multi-agent properties when vehicles are driving in traffic. Such models lead to deficiencies in interaction and collaboration between vehicles, which results in hazardous driving behaviors and overall traffic inefficiency. In this paper, we revisit the lane-change problem and propose a bi-level lane-change behavior planning strategy, where the upper level is a novel multi-agent deep reinforcement learning (DRL) based lane-change decision model and the lower level is a negotiation based right-of-way assignment model. We promote the collaboration performance of the upper-level lane-change decision model from three crucial aspects. First, we formulate the lane-change decision problem with a novel multi-agent reinforcement learning model, which provides a more appropriate paradigm for collaboration than the single-agent model. Second, we encode the driving intentions of surrounding vehicles into the observation space, which can empower multiple vehicles to implicitly negotiate the right-of-way in decision-making and enable the model to determine the right-of-way in a collaborative manner. Third, an ingenious reward function is designed to allow the vehicles to consider not only ego benefits but also the impact of changing lanes on traffic, which will guide the multi-agent system to learn excellent coordination performance. With the upper-level lane-change decisions, the lower-level right-of-way assignment model is used to guarantee the safety of lane-change behaviors. The experiments show that the proposed approaches can lead to safe, efficient, and harmonious lane-change behaviors, which boosts the collaboration between vehicles and in turn improves the safety and efficiency of the overall traffic. Moreover, the proposed approaches promote the microscopic synchronization of vehicles, which can lead to the macroscopic synchronization of traffic flow.

Learning to Change: Choreographing Mixed Traffic Through Lateral Control and Hierarchical Reinforcement Learning

Learning to control and coordinate mixed traffic through robot vehicles at complex and unsignalized intersections

Heterogeneous Mixed Traffic Control and Coordination

Managing Mixed Traffic at Signalized Intersections: an Adaptive Signal Control and CAV Coordination System Based on Deep Reinforcement Learning.

Neighbor-Aware Reinforcement Learning for Mixed Traffic Optimization in Large-scale Networks

Heterogeneous Traffic Intersections Control Design Based on Reinforcement Learning

An Improved Hierarchical Deep Reinforcement Learning Algorithm for Multi-Intelligent Vehicle Lane Change

A Control Method with Reinforcement Learning for Urban Un-Signalized Intersection in Hybrid Traffic Environment

Hybrid Reinforcement Learning-Based Eco-Driving Strategy for Connected and Automated Vehicles at Signalized Intersections

Traffic Signal Control in Mixed Traffic Environment Based on Advance Decision and Reinforcement Learning

Hierarchical Reinforcement Learning for Dynamic Autonomous Vehicle Navigation at Intelligent Intersections

Courteous Behavior of Automated Vehicles at Unsignalized Intersections Via Reinforcement Learning

Deep Reinforcement Learning for Traffic Signal Control along Arterials

Integrated decision and control at multi-lane intersections with mixed traffic flow

Multi-Agent DRL-Based Lane Change with Right-of-Way Collaboration Awareness

A Hierarchical Spatio-Temporal Cooperative Reinforcement Learning Approach for Traffic Signal Control.

Bi-Level Control of Weaving Sections in Mixed Traffic Environments with Connected and Automated Vehicles

Connected Automated Vehicle Cooperative Control with a Deep Reinforcement Learning Approach in a Mixed Traffic Environment

An Efficient Deep Reinforcement Learning Model for Urban Traffic Control

Adaptive Road Configurations for Improved Autonomous Vehicle-Pedestrian Interactions using Reinforcement Learning