Abstract:Distributedly training models across diverse clients with heterogeneous data samples can significantly impact the convergence of federated learning. Various novel federated learning methods address these challenges but often require significant communication resources and local computational capacity, leading to reduced global inference accuracy in scenarios with imbalanced label data distribution and quantity skew. To tackle these challenges, we propose FedVGL, a Federated Variational Generative Learning method that directly trains a local generative model to learn the distribution of local features and improve global target model inference accuracy during aggregation, particularly under conditions of severe data heterogeneity. FedVGL facilitates distributed learning by sharing generators and latent vectors with the global server, aiding in global target model training from mapping local data distribution to the variational latent space for feature reconstruction. Additionally, FedVGL implements anonymization and encryption techniques to bolster privacy during generative model transmission and aggregation. In comparison to vanilla federated learning, FedVGL minimizes communication overhead, demonstrating superior accuracy even with minimal communication rounds. It effectively mitigates model drift in scenarios with heterogeneous data, delivering improved target model training outcomes. Empirical results establish FedVGL's superiority over baseline federated learning methods under severe label imbalance and data skew condition. In a Label-based Dirichlet Distribution setting with α =0.01 and 10 clients using the MNIST dataset, FedVGL achieved an exceptional accuracy over 97% with the VGG-9 target model.

Variational Federated Multi-Task Learning

HFedMTL: Hierarchical Federated Multi-Task Learning

Efficient Federated Multi-view Learning

Multi-Model Federated Learning

Federated Variational Generative Learning for Heterogeneous Data in Distributed Environments

How to Combine Variational Bayesian Networks in Federated Learning

Flexible Vertical Federated Learning with Heterogeneous Parties

Decoupled Vertical Federated Learning for Practical Training on Vertically Partitioned Data

Scalable Vertical Federated Learning via Data Augmentation and Amortized Inference

FedMVT: Semi-supervised Vertical Federated Learning with MultiView Training.

Federated Multi-Task Learning under a Mixture of Distributions

Variational Bayes for Federated Continual Learning

Asynchronous Multi-Server Federated Learning for Geo-Distributed Clients

Multi-VFL: A Vertical Federated Learning System for Multiple Data and Label Owners

Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descent

EFMVFL: An Efficient and Flexible Multi-party Vertical Federated Learning without a Third Party

Clustered Federated Learning via Generalized Total Variation Minimization

Achieving Model Fairness in Vertical Federated Learning

Federated Learning with Data-Agnostic Distribution Fusion

Active-Passive Federated Learning for Vertically Partitioned Multi-view Data