Abstract:Mobile devices, including smartphones and laptops, generate decentralized and heterogeneous data, presenting significant challenges for traditional centralized machine learning models due to substantial communication costs and privacy risks. Federated Learning (FL) offers a promising alternative by enabling collaborative training of a global model across decentralized devices without data sharing. However, FL faces challenges due to statistical heterogeneity among clients, where non-independent and identically distributed (non-IID) data impedes model convergence and performance. This paper focuses on data-dependent heterogeneity in FL and proposes a novel approach leveraging mean latent representations extracted from locally trained models. The proposed method normalizes client contributions based on these representations, allowing the central server to estimate and adjust for heterogeneity during aggregation. This normalization enhances the global model's generalization and mitigates the limitations of conventional federated averaging methods. The main contributions include introducing a normalization scheme using mean latent representations to handle statistical heterogeneity in FL, demonstrating the seamless integration with existing FL algorithms to improve performance in non-IID settings, and validating the approach through extensive experiments on diverse datasets. Results show significant improvements in model accuracy and consistency across skewed distributions. Our experiments with six FL schemes: FedAvg, FedProx, FedBABU, FedNova, SCAFFOLD, and SGDM highlight the robustness of our approach. This research advances FL by providing a practical and computationally efficient solution for statistical heterogeneity, contributing to the development of more reliable and generalized machine learning models.

Flexible Contribution Estimation Methods for Horizontal Federated Learning.

Contributions Estimation in Federated Learning: A Comprehensive Experimental Evaluation

Efficient Client Contribution Evaluation for Horizontal Federated Learning

Evaluate the Contribution of Multiple Participants in Federated Learning

Fast, Robust and Interpretable Participant Contribution Estimation for Federated Learning

A Fair Contribution Measurement Method for Federated Learning

Contribution Evaluation in Federated Learning: Examining Current Approaches

Intrinsic Performance Influence-based Participant Contribution Estimation for Horizontal Federated Learning.

Contribution Evaluation of Heterogeneous Participants in Federated Learning via Prototypical Representations

Efficient Participant Contribution Evaluation for Horizontal and Vertical Federated Learning

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Towards Fair, Robust and Efficient Client Contribution Evaluation in Federated Learning

Measure Contribution of Participants in Federated Learning

FedEmb: A Vertical and Hybrid Federated Learning Algorithm using Network And Feature Embedding Aggregation

FairFed: Improving Fairness and Efficiency of Contribution Evaluation in Federated Learning via Cooperative Shapley Value

Contribution Matching-Based Hierarchical Incentive Mechanism Design for Crowd Federated Learning

Disentangling Client Contributions: Improving Federated Learning Accuracy in the Presence of Heterogeneous Data

A Survey on Contribution Evaluation in Vertical Federated Learning

Client Contribution Normalization for Enhanced Federated Learning

CDFed: Contribution-based Dynamic Federated Learning for Managing System and Statistical Heterogeneity

FedLaw: Value-Aware Federated Learning with Individual Fairness and Coalition Stability