Abstract:Federated Learning has sparked increasing interest as a promising approach to utilize large amounts of data stored on network edge devices. Federated Averaging is the most widely accepted Federated Learning framework. In Federated Averaging, the server keeps waiting for client models to compute the global model in each round unless all client models are received or a pre-configured timer expires, therefore it suffers seriously from participant devices with weak computation and/or communication capability, which is a kind of straggler problem. In this paper we design FedPA, a framework based on partial model aggregation strategy, in which the server waits for only an appropriate number of device models (referred to as aggregation number) in each round. Our experiment shows that the accuracy loss of the aggregated global model in a single round is not significant if the aggregation number is decided carefully. We propose a waiting strategy to determine the aggregation number for each round dynamically and the aggregation number is adaptive to achieve a tradeoff between single-round training time and the expected number of rounds to reach the target accuracy. Stale models are also included during aggregation when they arrive, and their positive value and negative effect are carefully evaluated and reflected in the aggregation strategy. Experiments show that FedPA outperforms the baseline strategy FedAvg and other three algorithms named FedAsync, FLANP and AD-SG. It can work well in all scenarios with different distributions of data samples (characterized by non-IID ratio) and computation/communication capability (characterized by level of heterogeneity) among devices. Experiments also show that FedPA is robust when a certain amount of noise is added into the input from clients for privacy concerns.

Bridging Model Heterogeneity in Federated Learning via Uncertainty-based Asymmetrical Reciprocity Learning

FedDGP: Disentangling Global and Personal Models for Federated Learning

APFed: Attention-based Personalized Federated Learning in Model Heterogeneity Scenario.

FedAPEN: Personalized Cross-silo Federated Learning with Adaptability to Statistical Heterogeneity

An Aggregation-Free Federated Learning for Tackling Data Heterogeneity

FedEmb: A Vertical and Hybrid Federated Learning Algorithm using Network And Feature Embedding Aggregation

FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning

FedPartial: Enabling Model-Heterogeneous Federated Learning Via Partial Model Transmission and Aggregation

Towards Personalized Federated Learning via Heterogeneous Model Reassembly

FedFed: Feature Distillation Against Data Heterogeneity in Federated Learning

FedUP: Bridging Fairness and Efficiency in Cross-Silo Federated Learning

FedTSA: A Cluster-based Two-Stage Aggregation Method for Model-heterogeneous Federated Learning

Few-Shot Model Agnostic Federated Learning

Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombination

Pfedsim: Similarity-Aware Model Aggregation Towards Personalized Federated Learning.

FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Update

FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation

FedHe: Heterogeneous Models and Communication-Efficient Federated Learning

Privacy preserving federated learning for full heterogeneity

Federated Mutual Learning: a Collaborative Machine Learning Method for Heterogeneous Data, Models, and Objectives

FedPA: An adaptively partial model aggregation strategy in Federated Learning