Abstract:Federated Learning using the Federated Averaging algorithm has shown great advantages for large-scale applications that rely on collaborative learning, especially when the training data is either unbalanced or inaccessible due to privacy constraints. We hypothesize that Federated Averaging underestimates the full extent of heterogeneity of data when the aggregation is performed. We propose Precision-weighted Federated Learning a novel algorithm that takes into account the variance of the stochastic gradients when computing the weighted average of the parameters of models trained in a Federated Learning setting. With Precision-weighted Federated Learning, we provide an alternate averaging scheme that leverages the heterogeneity of the data when it has a large diversity of features in its composition. Our method was evaluated using standard image classification datasets with two different data partitioning strategies (IID/non-IID) to measure the performance and speed of our method in resource-constrained environments, such as mobile and IoT devices. We obtained a good balance between computational efficiency and convergence rates with Precision-weighted Federated Learning. Our performance evaluations show 9% better predictions with MNIST, 18% with Fashion-MNIST, and 5% with CIFAR-10 in the non-IID setting. Further reliability evaluations ratify the stability in our method by reaching a 99% reliability index with IID partitions and 96% with non-IID partitions. In addition, we obtained a 20x speedup on Fashion-MNIST with only 10 clients and up to 37x with 100 clients participating in the aggregation concurrently per communication round. The results indicate that Precision-weighted Federated Learning is an effective and faster alternative approach for aggregating private data, especially in domains where data is highly heterogeneous.

FedExg: Federated Learning with Model Exchange

FedDGP: Disentangling Global and Personal Models for Federated Learning

Cross-Domain Federated Data Modeling on Non-IID Data

FedEx: Expediting Federated Learning over Heterogeneous Mobile Devices by Overlapping and Participant Selection

Communication-Efficient and Privacy-Preserving Large-Scale Federated Learning Counteracting Heterogeneity

FedMe: Federated Learning via Model Exchange

Few-Shot Model Agnostic Federated Learning

Lazy Aggregation for Heterogeneous Federated Learning

FedFed: Feature Distillation Against Data Heterogeneity in Federated Learning

FedXGBoost: Privacy-Preserving XGBoost for Federated Learning

Federated Learning Model Aggregation in Heterogenous Aerial and Space Networks

MpFedcon : Model-Contrastive Personalized Federated Learning with the Class Center

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation

FedAA: Using Non-sensitive Modalities to Improve Federated Learning while Preserving Image Privacy

FedMD: Heterogenous Federated Learning via Model Distillation

Precision-Weighted Federated Learning

FedEP: Tailoring Attention to Heterogeneous Data Distribution with Entropy Pooling for Decentralized Federated Learning

FedGKD: Towards Heterogeneous Federated Learning via Global Knowledge Distillation

Federated Learning Optimization: A Comparative Study of Data and Model Exchange Strategies in Dynamic Networks

Pfedbea: Combatting Data Heterogeneity for Personalized Federated Learning by Body Exchange and Aggregation Abandon