Abstract:The technology of dynamic map fusion among networked vehicles has been developed to enlarge sensing ranges and improve sensing accuracies for individual vehicles. This paper proposes a federated learning (FL) based dynamic map fusion framework to achieve high map quality despite unknown numbers of objects in fields of view (FoVs), various sensing and model uncertainties, and missing data labels for online learning. The novelty of this work is threefold: (1) developing a three-stage fusion scheme to predict the number of objects effectively and to fuse multiple local maps with fidelity scores; (2) developing an FL algorithm which fine-tunes feature models (i.e., representation learning networks for feature extraction) distributively by aggregating model parameters; (3) developing a knowledge distillation method to generate FL training labels when data labels are unavailable. The proposed framework is implemented in the Car Learning to Act (CARLA) simulation platform. Extensive experimental results are provided to verify the superior performance and robustness of the developed map fusion and FL schemes.

What problem does this paper attempt to address?

This paper attempts to solve the dynamic map fusion problem in the Intelligent Networked Vehicle System (INVS). Specifically: 1. **Reduce perception uncertainty**: Due to the limitations of sensors and the complexity of the environment, the perception data of a single vehicle may be missing, noisy or incorrect. By using the redundant measurement data provided by other vehicles, these uncertainties can be effectively reduced. 2. **Reduce model uncertainty**: Building a dynamic map on a vehicle or an edge server depends on the quality of the feature models of moving objects. These feature models are usually trained in the cloud using labeled datasets, but may not be fully optimized due to various factors. Therefore, a method is needed to improve the quality of these models. 3. **Lack of labels in online learning**: Although the datasets used in the cloud are manually labeled, the new data samples collected on vehicles are usually unlabeled. In order to achieve high - quality dynamic map updates, these local perception data samples need to be automatically labeled. To solve the above problems, the paper proposes a dynamic map fusion framework based on Federated Learning (FL). Its main contributions include: 1. **Three - stage fusion scheme**: It includes Density - Based Spatial Clustering of Applications with Noise (DBSCAN), score - based weighted average and Intersection over Union (IoU) - based box pruning to achieve global map fusion on the edge server. 2. **Distributed FL algorithm for point cloud data**: Fine - tune the feature models of moving objects in a distributed manner by aggregating model parameters. 3. **Knowledge Distillation (KD) method**: It is used to generate FL training labels without data labels, which is achieved through knowledge transfer from the edge server to each vehicle. 4. **Implementation on the CARLA simulation platform**: The superior performance and robustness of the proposed framework are verified through extensive test experiments, and open - source code is provided. In summary, this paper aims to solve the problems of perception uncertainty, model uncertainty and lack of labels in dynamic map fusion in the Intelligent Networked Vehicle System by combining Federated Learning and Knowledge Distillation techniques, so as to achieve high - quality map fusion and low communication overhead.

Distributed Dynamic Map Fusion via Federated Learning for Intelligent Networked Vehicles

Federated Deep Learning Meets Autonomous Vehicle Perception: Design and Verification

Mobility, Communication and Computation Aware Federated Learning for Internet of Vehicles

A Distributed Aggregation Approach for Vehicular Federated Learning

Federated Learning with Heterogeneous Data Handling for Robust Vehicular Object Detection

A Survey of Federated Learning for Connected and Automated Vehicles

FedRAV: Hierarchically Federated Region-Learning for Traffic Object Classification of Autonomous Vehicles

On the Federated Learning Framework for Cooperative Perception

Online Temporal Fusion for Vectorized Map Construction in Mapless Autonomous Driving

On Addressing Heterogeneity in Federated Learning for Autonomous Vehicles Connected to a Drone Orchestrator

MOB-FL: Mobility-Aware Federated Learning for Intelligent Connected Vehicles.

Mobility-Aware Federated Self-supervised Learning in Vehicular Network

Enabling Efficient Deep Convolutional Neural Network-based Sensor Fusion for Autonomous Driving

Federated Learning in Intelligent Transportation Systems: Recent Applications and Open Problems

Multi-Modality Cascaded Fusion Technology for Autonomous Driving

Semi-Interpenetrating Cooperative Localization in Connected Vehicle Networks

Privacy protection in intelligent vehicle networking: A novel federated learning algorithm based on information fusion

Clustered Vehicular Federated Learning: Process and Optimization

Hierarchical Decentralized Federated Learning Framework with Adaptive Clustering: Bloom-Filter-Based Companions Choice for Learning Non-IID Data in IoV

Federated Learning for Connected and Automated Vehicles: A Survey of Existing Approaches and Challenges