Abstract:Federated learning is a distributed machine learning framework. It facilitates collaborative modeling among participants without sharing raw user data, providing a feasible solution to address data silos in a secure and privacy-preserving manner. However, data heterogeneity is a major challenge of federated learning as it greatly impacts both the convergence speed during model training and the accuracy of predictions. To address this issue, recent federated learning algorithms have incorporated knowledge distillation as a approach of information sharing. Nevertheless, the majority of current approaches only employ logits averaging to combine participants' knowledge, which can have a detrimental impact on the accuracy of the global model, particularly when certain local models show poor performance. besides, some methods rely on public datasets, thereby compromising the privacy-preserving principle of federated learning. To address the aforementioned concerns, we introduces a new federated learning algorithm named Federated Adaptive Knowledge Distillation (FedAdKD), which utilizes knowledge distillation. After the server completes the aggregation of the global model, FedAdKD dynamically allocates distillation weights based on the individual local model's loss on the public dataset, and performs knowledge distillation on the global model. As the use of public datasets violates the privacy-preserving principle of federated learning, we also proposes the incorporation of generative models to generate data that adheres to the original image distribution. By employing federated learning on the client side to train diffusion models, data is generated that adheres to the original image distribution while maintaining privacy. This generated data is then utilized as the distillation dataset. Experimental findings validate the effectiveness of FedAdKD in addressing the obstacles presented by data heterogeneity. FedAdKD not only mitigates the decline in global model accuracy caused by subpar local models and minimizes knowledge forgetting resulting from direct model aggregation but also improves the generalization capacity of the global model.

Heterogeneous Defect Prediction Based on Federated Transfer Learning via Knowledge Distillation

Heterogeneous Defect Prediction with Two-Stage Ensemble Learning

Cost-sensitive transfer kernel canonical correlation analysis for heterogeneous defect prediction

Moving from Cross-Project Defect Prediction to Heterogeneous Defect Prediction: A Partial Replication Study

Landmark-Based Domain Adaptation and Selective Pseudo-Labeling for Heterogeneous Defect Prediction

Fedadkd:heterogeneous federated learning via adaptive knowledge distillation

Collective transfer learning for defect prediction

DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learning

Heterogeneous Federated Domain Generalization Network With Common Representation Learning for Cross-Load Machinery Fault Diagnosis

Bearing Faulty Prediction Method Based on Federated Transfer Learning and Knowledge Distillation

Privacy-preserving federated transfer learning for defect identification from highly imbalanced image data in additive manufacturing

FedTKD: A Trustworthy Heterogeneous Federated Learning Based on Adaptive Knowledge Distillation

A Prototype-Based Knowledge Distillation Framework for Heterogeneous Federated Learning

Unsupervised Deep Domain Adaptation for Heterogeneous Defect Prediction

Multiview Transfer Learning for Software Defect Prediction.

FedRAD: Heterogeneous Federated Learning via Relational Adaptive Distillation

SDP-MTF: A Composite Transfer Learning and Feature Fusion for Cross-Project Software Defect Prediction

Clustering Federated Learning for Wafer Defects Classification on Statistical Heterogeneous Data

Federated Transfer Learning in Fault Diagnosis under Data Privacy with Target Self-Adaptation

FedMD: Heterogenous Federated Learning via Model Distillation

DeFedTL: A Decentralized Federated Transfer Learning Method for Fault Diagnosis