Connected and Automated Vehicles in Mixed-Traffic: Learning Human Driver Behavior for Effective On-Ramp Merging

Nishanth Venkatesh,Viet-Anh Le,Aditya Dave,Andreas A. Malikopoulos

2023-04-02

Abstract:Highway merging scenarios featuring mixed traffic conditions pose significant modeling and control challenges for connected and automated vehicles (CAVs) interacting with incoming on-ramp human-driven vehicles (HDVs). In this paper, we present an approach to learn an approximate information state model of CAV-HDV interactions for a CAV to maneuver safely during highway merging. In our approach, the CAV learns the behavior of an incoming HDV using approximate information states before generating a control strategy to facilitate merging. First, we validate the efficacy of this framework on real-world data by using it to predict the behavior of an HDV in mixed traffic situations extracted from the Next-Generation Simulation repository. Then, we generate simulation data for HDV-CAV interactions in a highway merging scenario using a standard inverse reinforcement learning approach. Without assuming a prior knowledge of the generating model, we show that our approximate information state model learns to predict the future trajectory of the HDV using only observations. Subsequently, we generate safe control policies for a CAV while merging with HDVs, demonstrating a spectrum of driving behaviors, from aggressive to conservative. We demonstrate the effectiveness of the proposed approach by performing numerical simulations.

Machine Learning,Artificial Intelligence,Systems and Control

What problem does this paper attempt to address?

The paper primarily addresses the challenges of automated vehicles (CAV) and human-driven vehicles (HDV) merging at highway ramps in mixed traffic flow. Specifically, the paper aims to solve the following key issues: 1. **Predicting Human Driving Behavior**: In mixed traffic conditions, CAVs need to understand and predict the behavior of HDVs to achieve safe and efficient ramp merging. Therefore, the study proposes a method to learn the behavior patterns of HDVs and generate control strategies for CAVs based on this information state. 2. **Control Strategy Generation**: The paper develops an iterative Model Predictive Control (MPC) algorithm that uses the learned information state model to generate control strategies for CAVs that are both safe and energy-efficient. 3. **Validation of the Prediction Framework**: The effectiveness of the prediction framework is validated using real-world datasets (such as the Next-Generation Simulation dataset) to ensure it can accurately predict HDV behavior. 4. **Comprehensive Validation of Learning and Control Methods**: Through extensive simulation tests (5000 times), the effectiveness of the combined learning and control method is validated under different driving styles (ranging from aggressive to conservative). In summary, the goal of this paper is to propose a method that can learn HDV behavior online and generate safe control strategies accordingly, enabling CAVs to perform safe and efficient ramp merging operations in a mixed traffic environment. This method not only considers safety but also takes energy efficiency into account.

Connected and Automated Vehicles in Mixed-Traffic: Learning Human Driver Behavior for Effective On-Ramp Merging

Safe Merging in Mixed Traffic with Confidence

Novel Decision-Making Strategy for Connected and Autonomous Vehicles in Highway On-Ramp Merging

Cooperative Merging Trajectory Optimization of Connected and Automated Vehicles in the Mixed Traffic: a Receding Horizon Control Approach

Active Learning with Dual Model Predictive Path-Integral Control for Interaction-Aware Autonomous Highway On-ramp Merging

Safety-Critical and Flexible Cooperative On-Ramp Merging Control of Connected and Automated Vehicles in Mixed Traffic

Driving Safety Field Modeling Focused on Heterogeneous Traffic Flows and Cooperative Control Strategy in Highway Merging Zone

Coordination for Connected Automated Vehicles at Merging Roadways in Mixed Traffic Environment

A Hierarchical Model-Based Optimization Control Approach for Cooperative Merging by Connected Automated Vehicles

Evaluation of Connected Vehicle Identification-Aware Mixed Traffic Freeway Cooperative Merging

A Deep Reinforcement Learning-Based Approach for Autonomous Driving in Highway On-Ramp Merge

Autonomous Highway Merging in Mixed Traffic Using Reinforcement Learning and Motion Predictive Safety Controller

Cooperative Ramp Merging for Mixed Traffic with Connected Automated Vehicles and Human-Operated Vehicles

Cooperative Merging for Multiple Connected and Automated Vehicles at Highway On-Ramps Via Virtual Platoon Formation

Hierarchical Optimal Maneuver Planning and Trajectory Control at On-Ramps With Multiple Mainstream Lanes

On Social Interactions of Merging Behaviors at Highway On-Ramps in Congested Traffic

Cooperative Merging Strategy for Connected and Automated Vehicles at Highway On-Ramps

Developing Merging Policies for CAVs: A Policy Training Framework Combining Human Experience with Reinforcement Learning

A Multi-Agent Deep Reinforcement Learning Coordination Framework for Connected and Automated Vehicles at Merging Roadways

Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic

Cooperative Highway Work Zone Merge Control Based on Reinforcement Learning in a Connected and Automated Environment