Abstract:Federated learning (FL) is emerging as a new privacy-preserving learning paradigm that allows multiple devices to collaborate in training a model without sharing their raw data, using a central server for coordination. However, device heterogeneity poses a challenge in FL, as participating devices often have different computing capacities. To address this issue, heterogeneous models need to be designed to accommodate different device computing capacities. The existing approach involves pre-designing multiple heterogeneous models and extracting sub-models from the server model. While such an approach effectively tackles device heterogeneity, it has several drawbacks such as high communication overhead and insufficient personalization. In each training round, the server distributes the entire model parameters to each device, and each device also transmits the entire model parameters to the server for aggregation. In this work, we propose FedPartial, a new framework that overcomes these challenges by introducing a partial model transmission and aggregation mechanism. The FedPartial framework eliminates the need for devices to transmit the entire model parameters in each training round while still benefiting from global model aggregation. Specifically, FedPartial divides the device model into two parts: the shallow part participates in the global aggregation of heterogeneous models, while the deep part remains on the device locally. By keeping the deep part of the model on the device, FedPartial reduces the communication overhead significantly and achieves a certain degree of personalization. Through extensive experiments, we demonstrate that FedPartial outperforms existing state-of-the-art methods, particularly in more complex and statistically heterogeneous scenarios.

FedOps: A Platform of Federated Learning Operations With Heterogeneity Management

A Platform of Federated Learning Management for Enhanced Mobile Collaboration

FlocOff: Data Heterogeneity Resilient Federated Learning with Communication-Efficient Edge Offloading

FedLPS: Heterogeneous Federated Learning for Multiple Tasks with Local Parameter Sharing

Federated Mutual Learning: a Collaborative Machine Learning Method for Heterogeneous Data, Models, and Objectives

FedML: A Research Library and Benchmark for Federated Machine Learning

Tackling Data Heterogeneity in Federated Learning via Loss Decomposition

FedH2L: A Federated Learning Approach with Model and Statistical Heterogeneity

FedHe: Heterogeneous Models and Communication-Efficient Federated Learning

Heterogeneous Federated Learning with Cross-layer Model Fusion.

Federated Loss Exploration for Improved Convergence on Non-IID Data

FedPartial: Enabling Model-Heterogeneous Federated Learning Via Partial Model Transmission and Aggregation

Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning

Heterogeneous Federated Learning via Grouped Sequential-to-Parallel Training

Completely Heterogeneous Federated Learning

HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients

FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation

FedShift: Tackling Dual Heterogeneity Problem of Federated Learning via Weight Shift Aggregation

Federated learning with workload-aware client scheduling in heterogeneous systems

FedBranch: Heterogeneous Federated Learning Via Multi-Branch Neural Network.

FLASH: Heterogeneity-Aware Federated Learning at Scale