Abstract:Federated learning, as a distributed machine learning framework, aims to protect data privacy while addressing the issue of data silos by collaboratively training models across multiple clients. However, a significant challenge to federated learning arises from the non-independent and identically distributed (non-iid) nature of data across different clients. non-iid data can lead to inconsistencies between the minimal loss experienced by individual clients and the global loss observed after the central server aggregates the local models, affecting the model's convergence speed and generalization capability. To address this challenge, we propose a novel federated learning algorithm based on update bias (FedUB). Unlike traditional federated learning approaches such as FedAvg and FedProx, which independently update model parameters on each client before direct aggregation to form a global model, the FedUB algorithm incorporates an update bias in the loss function of local models—specifically, the difference between each round's local model updates and the global model updates. This design aims to reduce discrepancies between local and global updates, thus aligning the parameters of locally updated models more closely with those of the globally aggregated model, thereby mitigating the fundamental conflict between local and global optima. Additionally, during the aggregation phase at the server side, we introduce a metric called the bias metric, which assesses the similarity between each client's local model and the global model. This metric adaptively sets the weight of each client during aggregation after each training round to achieve a better global model. Extensive experiments conducted on multiple datasets have confirmed the effectiveness of the FedUB algorithm. The results indicate that FedUB generally outperforms methods such as FedDC, FedDyn, and Scaffold, especially in scenarios involving partial client participation and non-iid data distributions. It demonstrates superior performance and faster convergence in tasks such as image classification.

FedDNA: Federated Learning with Decoupled Normalization-Layer Aggregation for Non-IID Data

Understanding the Training Dynamics in Federated Deep Learning via Aggregation Weight Optimization

FedDGP: Disentangling Global and Personal Models for Federated Learning

Federated Learning with Data-Agnostic Distribution Fusion

AsyncFedED: Asynchronous Federated Learning with Euclidean Distance Based Adaptive Weight Aggregation

FedAgg: Adaptive Federated Learning with Aggregated Gradients

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

FedSiam-DA: Dual-aggregated Federated Learning Via Siamese Network for Non-Iid Data

FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction

FedVeca: Federated Vectorized Averaging on Non-IID Data with Adaptive Bi-directional Global Objective

Towards addressing aggregation deviation for model training in resource-scarce edge environment

Federated Learning on Non-Independent and Identically Distributed Data

FedPA: An adaptively partial model aggregation strategy in Federated Learning

FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning

Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombination

Learning to aggregate: A parameterized aggregator to debias aggregation for cross-device federated learning

Collaboratively Learning Federated Models from Noisy Decentralized Data

An Aggregation-Free Federated Learning for Tackling Data Heterogeneity

FedBN: Federated Learning on Non-IID Features via Local Batch Normalization

FedDA: Resource-adaptive Federated Learning with Dual-Alignment Aggregation Optimization for Heterogeneous Edge Devices

FedUB: Federated Learning Algorithm Based on Update Bias