Abstract:In federated learning, non-iid data not only diminishes the performance of the global model but also gives rise to the fairness problem which manifests as an increase in the variance of the global model’s accuracy across clients. Fairness issues can result in the global model performing poorly or even failing on certain clients. Existing methods addressing the fairness problem in federated learning tend to neglect the comprehensive improvement of both the average performance and fairness of the global model. In addressing it, the multi-objective optimization method for fine-tuning global gradients, FedMC algorithm is introduced in this paper. The primary objective is the average loss function of all clients, and the sub-objective involves fine-tuning the global gradient by reducing the gradient conflict between the global gradient and the local gradients. Specifically, we refine the global gradient by incorporating a sub-optimization objective aimed at alleviating conflicts between the global gradient and the local gradient with the largest deviation, denoted as FedMC. FedMC can enhance the performance and convergence rate of clients with initially poor performance, albeit at the cost of the earlier convergence rate of clients with initially good performance. Nevertheless, it enables the latter to reach the accuracy level achieved before fine-tuning. In addition, we also propose FedMC+ algorithm, owning three additional optimization mechanisms built upon the FedMC optimization objective which includes the decay of hyperparameter, the sliding window mechanism, and data-balanced client selection. Besides, we present a theoretical analysis of the convergence rate of FedMC, demonstrating its convergence to a Pareto stationary solution. Our combined experimental results confirm that FedMC+ achieves an average 4.5% improvement in accuracy and a 22% reduction in the degree of dispersion compared to state-of-the-art federated learning (FL) methods.

An Experimental Study of Class Imbalance in Federated Learning

A Survey on Class Imbalance in Federated Learning

Addressing Class Imbalance in Federated Learning

Learning by Imitating the Classics: Mitigating Class Imbalance in Federated Learning Via Simulated Centralized Learning

Class-Imbalance and Client-Imbalance Federated Learning for Fault Diagnosis

Federated Learning with Classifier Shift for Class Imbalance.

Aligning Model Outputs for Class Imbalanced Non-Iid Federated Learning

Performance Enhancement in Federated Learning by Reducing Class Imbalance of Non-IID Data

Multi-Objective Federated Learning: Balancing Global Performance and Individual Fairness

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Integrating Local Real Data with Global Gradient Prototypes for Classifier Re-Balancing in Federated Long-Tailed Learning

Understanding the Training Dynamics in Federated Deep Learning via Aggregation Weight Optimization

Addressing Class Variable Imbalance in Federated Semi-supervised Learning

Adaptive Clustering-Based Model Aggregation for Federated Learning with Imbalanced Data

FedDistill: Global Model Distillation for Local Model De-Biasing in Non-IID Federated Learning

BalanceFL: Addressing Class Imbalance in Long-Tail Federated Learning

Exploring Amplified Heterogeneity Arising from Heavy-Tailed Distributions in Federated Learning

FedCD: A Classifier Debiased Federated Learning Framework for Non-IID Data

The Impact of Differential Privacy on Model Fairness in Federated Learning.

Anti-Matthew FL: Bridging the Performance Gap in Federated Learning to Counteract the Matthew Effect

Privacy, accuracy, and model fairness trade-offs in federated learning