Abstract:One-shot federated learning (OSFL) has gained attention in recent years due to its low communication cost. However, most of the existing methods require auxiliary datasets or training generators, which hinders their practicality in real-world scenarios. In this paper, we explore the novel opportunities that diffusion models bring to OSFL and propose FedCADO, utilizing guidance from client classifiers to generate data that complies with clients' distributions and subsequently training the aggregated model on the server. Specifically, our method involves targeted optimizations in two aspects. On one hand, we conditionally edit the randomly sampled initial noises, embedding them with specified semantics and distributions, resulting in a significant improvement in both the quality and stability of generation. On the other hand, we employ the BN statistics from the classifiers to provide detailed guidance during generation. These tailored optimizations enable us to limitlessly generate datasets, which closely resemble the distribution and quality of the original client dataset. Our method effectively handles the heterogeneous client models and the problems of non-IID features or labels. In terms of privacy protection, our method avoids training any generator or transferring any auxiliary information on clients, eliminating any additional privacy leakage risks. Leveraging the extensive knowledge stored in the pre-trained diffusion model, the synthetic datasets can assist us in surpassing the knowledge limitations of the client samples, resulting in aggregation models that even outperform the performance ceiling of centralized training in some cases, which is convincingly demonstrated in the sufficient quantification and visualization experiments conducted on three large-scale multi-domain image datasets.

One-shot Federated Learning via Synthetic Distiller-Distillate Communication

FedDistill: Global Model Distillation for Local Model De-Biasing in Non-IID Federated Learning

Unlocking the Potential of Federated Learning: The Symphony of Dataset Distillation via Deep Generative Latents

DFDG: Data-Free Dual-Generator Adversarial Distillation for One-Shot Federated Learning

Federated Virtual Learning on Heterogeneous Data with Local-global Distillation

One-shot Federated Learning on Medical Data using Knowledge Distillation with Image Synthesis and Client Model Adaptation

A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning

One-Shot Federated Learning with Classifier-Guided Diffusion Models

Communication-Efficient Federated Distillation with Active Data Sampling

FedFed: Feature Distillation Against Data Heterogeneity in Federated Learning

FedMD: Heterogenous Federated Learning via Model Distillation

DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learning

Ensemble Distillation for Robust Model Fusion in Federated Learning

Fine-tuning Global Model Via Data-Free Knowledge Distillation for Non-IID Federated Learning

Data-Free Knowledge Distillation for Heterogeneous Federated Learning

Enhancing Federated Learning by One-Shot Transferring of Intermediate Features from Clients

DENSE: Data-Free One-Shot Federated Learning

One-shot Federated Learning Without Server-side Training

Communication-Efficient and Attack-Resistant Federated Edge Learning with Dataset Distillation

To Distill or Not To Distill: Towards Fast, Accurate and Communication Efficient Federated Distillation Learning

Federated Distillation: A Survey