Abstract:Integrating large language models (LLMs) into autonomous driving enhances personalization and adaptability in open-world scenarios. However, traditional edge computing models still face significant challenges in processing complex driving data, particularly regarding real-time performance and system efficiency. To address these challenges, this study introduces EC-Drive, a novel edge-cloud collaborative autonomous driving system with data drift detection capabilities. EC-Drive utilizes drift detection algorithms to selectively upload critical data, including new obstacles and traffic pattern changes, to the cloud for processing by GPT-4, while routine data is efficiently managed by smaller LLMs on edge devices. This approach not only reduces inference latency but also improves system efficiency by optimizing communication resource use. Experimental validation confirms the system's robust processing capabilities and practical applicability in real-world driving conditions, demonstrating the effectiveness of this edge-cloud collaboration framework. Our data and system demonstration will be released at <a class="link-external link-https" href="https://sites.google.com/view/ec-drive" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the challenges of real - time performance and system efficiency faced by traditional edge - computing models when processing complex driving data in autonomous driving. Specifically: 1. **Challenges of real - time performance**: Traditional edge - computing models have difficulty in ensuring low - latency inference speed when processing complex driving data, especially in the face of dynamically changing traffic environments. 2. **Challenges of system efficiency**: Due to the limited resources of edge devices, their ability to process large - scale data and complex tasks is insufficient, resulting in low overall system efficiency. To solve these problems, the paper proposes a new edge - cloud - collaborative autonomous driving system - EC - Drive, which has the following features: - **Data drift detection ability**: Through the data drift detection algorithm, EC - Drive can identify the key data (such as new obstacles, changes in traffic patterns, etc.) that need to be uploaded to the cloud for processing, thereby reducing unnecessary data transmission and improving the utilization rate of communication resources. - **Edge - cloud collaboration mechanism**: Simple tasks are processed by small - scale language models (LLMs) on edge devices, and complex tasks are uploaded to the cloud to be processed by large - scale language models (such as GPT - 4). This way of division of labor not only reduces inference latency but also improves the overall efficiency and adaptability of the system. ### Specific solutions 1. **Edge - cloud collaborative architecture**: - **Edge devices**: Deploy small - scale language models (such as LLaMA - Adapter) for real - time processing of routine driving data and making preliminary decisions. - **Cloud**: Deploy large - scale language models (such as GPT - 4) for in - depth analysis and optimized decision - making in complex scenarios. 2. **Data drift detection**: - Use the Alibi Detect library to monitor the performance of edge models. When an anomaly or low - confidence prediction is detected, the relevant data is uploaded to the cloud for further processing. 3. **Multimodal fusion**: - Combine language features with visual data to enhance the interpretability and decision - making ability of the system, enabling it to better understand and respond to new traffic situations. Through these methods, the EC - Drive system can effectively respond to complex and changeable driving environments while ensuring low latency, improving the safety and reliability of the autonomous driving system. Experimental verification shows that this system performs excellently in actual driving scenarios, demonstrating the practical application potential of the edge - cloud collaborative framework.

Edge-Cloud Collaborative Motion Planning for Autonomous Driving with Large Language Models

Efficient Driving Behavior Narration and Reasoning on Edge Device Using Large Language Models

A Cloud-Edge Collaborative Architecture for Multimodal LLMs-Based Advanced Driver Assistance Systems in IoT Networks

LLM4Drive: A Survey of Large Language Models for Autonomous Driving

Asynchronous Large Language Model Enhanced Planner for Autonomous Driving

Large Language Models Powered Context-aware Motion Prediction in Autonomous Driving

Personalized Autonomous Driving with Large Language Models: Field Experiments

Intelligent Driving Task Scheduling Service in Vehicle-Edge Collaborative Networks Based on Deep Reinforcement Learning

DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving

Deep Learning Enhanced Driving Behavior Evaluation Based on Vehicle-Edge-Cloud Architecture

DriveLLM: Charting the Path Toward Full Autonomous Driving with Large Language Models

LMDrive: Closed-Loop End-to-End Driving with Large Language Models

Edge Intelligence for Autonomous Driving in 6G Wireless System: Design Challenges and Solutions

Deep Reinforcement Learning for Collaborative Edge Computing in Vehicular Networks

Cloud-Edge-Terminal Collaborative AIGC for Autonomous Driving

LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving

LAECIPS: Large Vision Model Assisted Adaptive Edge-Cloud Collaboration for IoT-based Perception System

DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model

Hierarchical Large Language Models in Cloud Edge End Architecture for Heterogeneous Robot Cluster Control

Edge-LLM: A Collaborative Framework for Large Language Model Serving in Edge Computing

Large Language Models for Autonomous Driving (LLM4AD): Concept, Benchmark, Simulation, and Real-Vehicle Experiment