Abstract:Federated learning is a machine learning paradigm that enables decentralized clients to collaboratively learn a shared model while keeping all the training data local. While considerable research has focused on federated image generation, particularly Generative Adversarial Networks, Variational Autoencoders have received less attention. In this paper, we address the challenges of non-IID (independently and identically distributed) data environments featuring multiple groups of images of different types. Specifically, heterogeneous data distributions can lead to difficulties in maintaining a consistent latent space and can also result in local generators with disparate texture features being blended during aggregation. We introduce a novel approach, FissionVAE, which decomposes the latent space and constructs decoder branches tailored to individual client groups. This method allows for customized learning that aligns with the unique data distributions of each group. Additionally, we investigate the incorporation of hierarchical VAE architectures and demonstrate the use of heterogeneous decoder architectures within our model. We also explore strategies for setting the latent prior distributions to enhance the decomposition process. To evaluate our approach, we assemble two composite datasets: the first combines MNIST and FashionMNIST; the second comprises RGB datasets of cartoon and human faces, wild animals, marine vessels, and remote sensing images of Earth. Our experiments demonstrate that FissionVAE greatly improves generation quality on these datasets compared to baseline federated VAE models.

High Fidelity Image Synthesis With Deep VAEs In Latent Space

3D-Aware Image Synthesis Via Learning Structural and Textural Representations

Generating Diverse High-Fidelity Images with VQ-VAE-2

NVAE: A Deep Hierarchical Variational Autoencoder

Wavelets to the Rescue: Improving Sample Quality of Latent Variable Deep Generative Models

DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents

Pyramid-VAE-GAN: Transferring Hierarchical Latent Variables for Image Inpainting

Multi-Stage Variational Auto-Encoders for Coarse-to-Fine Image Generation

CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training

Super-resolution Variational Auto-Encoders

TimeVAE: A Variational Auto-Encoder for Multivariate Time Series Generation

Multimodal hierarchical Variational AutoEncoders with Factor Analysis latent space

Exploiting Latent Codes: Interactive Fashion Product Generation, Similar Image Retrieval, and Cross-Category Recommendation using Variational Autoencoders

Optimizing Hierarchical Image VAEs for Sample Quality

DualVAE: Controlling Colours of Generated and Real Images

Hybrid VAE: Improving Deep Generative Models using Partial Observations

FissionVAE: Federated Non-IID Image Generation with Latent Space and Decoder Decomposition

Deep Feature Consistent Variational Autoencoder

Diagnosing and Enhancing VAE Models

LiteVAE: Lightweight and Efficient Variational Autoencoders for Latent Diffusion Models

Image generation via latent space learning using improved combination