Abstract:Federated learning (FL) has shown remarkable success in cooperatively training deep models, while typically struggling with noisy labels. Advanced works propose to tackle label noise by a re-weighting strategy with a strong assumption, i.e., mild label noise. However, it may be violated in many real-world FL scenarios because of highly contaminated clients, resulting in extreme noise ratios, e.g., $>$90%. To tackle extremely noisy clients, we study the robustness of the re-weighting strategy, showing a pessimistic conclusion: minimizing the weight of clients trained over noisy data outperforms re-weighting strategies. To leverage models trained on noisy clients, we propose a novel approach, called negative distillation (FedNed). FedNed first identifies noisy clients and employs rather than discards the noisy clients in a knowledge distillation manner. In particular, clients identified as noisy ones are required to train models using noisy labels and pseudo-labels obtained by global models. The model trained on noisy labels serves as a `bad teacher' in knowledge distillation, aiming to decrease the risk of providing incorrect information. Meanwhile, the model trained on pseudo-labels is involved in model aggregation if not identified as a noisy client. Consequently, through pseudo-labeling, FedNed gradually increases the trustworthiness of models trained on noisy clients, while leveraging all clients for model aggregation through negative distillation. To verify the efficacy of FedNed, we conduct extensive experiments under various settings, demonstrating that FedNed can consistently outperform baselines and achieve state-of-the-art performance. Our code is available at <a class="link-external link-https" href="https://github.com/linChen99/FedNed" rel="external noopener nofollow">this https URL</a>.

Robust Federated Learning with Noisy and Heterogeneous Clients

Learning Cautiously in Federated Learning with Noisy and Heterogeneous Clients

Privacy preserving federated learning for full heterogeneity

Completely Heterogeneous Federated Learning

Advances in Robust Federated Learning: Heterogeneity Considerations

Robust Federated Learning for Heterogeneous Model and Data

Collaboratively Learning Federated Models from Noisy Decentralized Data

Robust Federated Opportunistic Learning in the Presence of Label Quality Disparity

Noise-Robust and Resource-Efficient ADMM-based Federated Learning

Combating Exacerbated Heterogeneity for Robust Models in Federated Learning

No One Left Behind: Inclusive Federated Learning over Heterogeneous Devices

Robust Hierarchical Federated Learning with Anomaly Detection in Cloud-Edge-End Cooperation Networks

Federated Learning with Extremely Noisy Clients via Negative Distillation

Communication Efficient Federated Learning With Heterogeneous Structured Client Models

FedRGL: Robust Federated Graph Learning for Label Noise

Critical care: Estimating renal function in the ICU: even more challenging

Continual Horizontal Federated Learning for Heterogeneous Data

Overhead-free Noise-tolerant Federated Learning: A New Baseline

Combating Exacerbated Heterogeneity for Robust Decentralized Models

HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients

Robust Federated Learning Through Representation Matching and Adaptive Hyper-parameters