Abstract:Discretionary lane-change is one of the critical challenges for autonomous vehicle (AV) design due to its significant impact on traffic efficiency. Existing intelligent lane-change solutions have primarily focused on optimizing the performance of the ego-vehicle, thereby suffering from limited generalization performance. Recent research has seen an increased interest in multi-agent reinforcement learning (MARL)-based approaches to address the limitation of the ego vehicle-based solutions through close coordination of multiple agents. Although MARL-based approaches have shown promising results, the potential impact of lane-change decisions on the overall traffic flow of a road segment has not been fully considered. In this paper, we present a novel hybrid MARL-based intelligent lane-change system for AVs designed to jointly optimize the local performance for the ego vehicle, along with the global performance focused on the overall traffic flow of a given road segment. With a careful review of the relevant transportation literature, a novel state space is designed to integrate both the critical local traffic information pertaining to the surrounding vehicles of the ego vehicle, as well as the global traffic information obtained from a road-side unit (RSU) responsible for managing a road segment. We create a reward function to ensure that the agents make effective lane-change decisions by considering the performance of the ego vehicle and the overall improvement of traffic flow. A multi-agent deep Q-network (DQN) algorithm is designed to determine the optimal policy for each agent to effectively cooperate in performing lane-change maneuvers. LCS-TF's performance was evaluated through extensive simulations in comparison with state-of-the-art MARL models. In all aspects of traffic efficiency, driving safety, and driver comfort, the results indicate that LCS-TF exhibits superior performance.

A heuristic-action-involved safe lane-change of autonomous vehicles with multiple constraints on road centerline and speed under highway environment

A Reinforcement Learning Approach to Smart Lane Changes of Self-driving Cars

A Lane Changing Time Point and Path Tracking Framework for Autonomous Ground Vehicle

Developing Smart Lane-changing Strategies for CAVs on Freeways based on MOBIL and Reinforcement Learning

Reinforcement Learning from Human Feedback for Lane Changing of Autonomous Vehicles in Mixed Traffic

A multi-agent reinforcement learning-based longitudinal and lateral control of CAVs to improve traffic efficiency in a mandatory lane change scenario

A Deep Reinforcement Learning-Based Approach for Autonomous Lane-Changing Velocity Control in Mixed Flow of Vehicle Group Level

Multi-agent reinforcement learning for cooperative lane changing of connected and autonomous vehicles in mixed traffic

Humanlike Decision and Motion Planning for Expressway Lane Changing Based on Artificial Potential Field

An Improved Hierarchical Deep Reinforcement Learning Algorithm for Multi-Intelligent Vehicle Lane Change

A Cooperation-Aware Lane Change Method for Automated Vehicles

A Cooperative Control Framework for CAV Lane Change in a Mixed Traffic Environment

Modeling and field experiments on autonomous vehicle lane changing with surrounding human-driven vehicles

Automatic Lane-Changing Trajectory Planning: from Self-Optimum to Local-Optimum.

An Evolutionary Learning Framework of Lane-Changing Control for Autonomous Vehicles at Freeway Off-Ramps

A Lane Changing Model of Automatic Driving Vehicle Based on Reinforcement Learning.

Towards Optimal Lane-changing Coordination of CAVs in Multi-lane Mixed Traffic Scenarios

Safe reinforcement learning for autonomous vehicles to make lane-change decisions: Constraint based on Incomplete Information Game Theory

LCS-TF: Multi-Agent Deep Reinforcement Learning-Based Intelligent Lane-Change System for Improving Traffic Flow

Research on Lane-Change Decision and Planning in Multilane Expressway Scenarios for Autonomous Vehicles

A Safe and Efficient Lane Change Decision-Making Strategy of Autonomous Driving Based on Deep Reinforcement Learning