Abstract:One-shot federated learning (OSFL) has gained attention in recent years due to its low communication cost. However, most of the existing methods require auxiliary datasets or training generators, which hinders their practicality in real-world scenarios. In this paper, we explore the novel opportunities that diffusion models bring to OSFL and propose FedCADO, utilizing guidance from client classifiers to generate data that complies with clients' distributions and subsequently training the aggregated model on the server. Specifically, our method involves targeted optimizations in two aspects. On one hand, we conditionally edit the randomly sampled initial noises, embedding them with specified semantics and distributions, resulting in a significant improvement in both the quality and stability of generation. On the other hand, we employ the BN statistics from the classifiers to provide detailed guidance during generation. These tailored optimizations enable us to limitlessly generate datasets, which closely resemble the distribution and quality of the original client dataset. Our method effectively handles the heterogeneous client models and the problems of non-IID features or labels. In terms of privacy protection, our method avoids training any generator or transferring any auxiliary information on clients, eliminating any additional privacy leakage risks. Leveraging the extensive knowledge stored in the pre-trained diffusion model, the synthetic datasets can assist us in surpassing the knowledge limitations of the client samples, resulting in aggregation models that even outperform the performance ceiling of centralized training in some cases, which is convincingly demonstrated in the sufficient quantification and visualization experiments conducted on three large-scale multi-domain image datasets.

A Practical Data-Free Approach to One-shot Federated Learning with Heterogeneity

One-shot Federated Learning via Synthetic Distiller-Distillate Communication

A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning

FedH2L: A Federated Learning Approach with Model and Statistical Heterogeneity

FedAPEN: Personalized Cross-silo Federated Learning with Adaptability to Statistical Heterogeneity

DENSE: Data-Free One-Shot Federated Learning

An Aggregation-Free Federated Learning for Tackling Data Heterogeneity

Enhancing Federated Learning by One-Shot Transferring of Intermediate Features from Clients

FedFed: Feature Distillation Against Data Heterogeneity in Federated Learning

One-Shot Federated Learning with Classifier-Guided Diffusion Models

One-Shot Sequential Federated Learning for Non-IID Data by Enhancing Local Model Diversity

Navigating Heterogeneity and Privacy in One-Shot Federated Learning with Diffusion Models

Completely Heterogeneous Federated Learning

Using Synthetic Data to Mitigate Unfairness and Preserve Privacy through Single-Shot Federated Learning

Privacy preserving federated learning for full heterogeneity

Fine-tuning Global Model Via Data-Free Knowledge Distillation for Non-IID Federated Learning

Improving Generalization and Personalization in Model-Heterogeneous Federated Learning

FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Models

Federated Virtual Learning on Heterogeneous Data with Local-global Distillation

Data-Free Knowledge Distillation for Heterogeneous Federated Learning

Towards Instance-adaptive Inference for Federated Learning