Abstract:This paper challenges the common assumption that the weight β, in β-VAE, should be larger than 1 in order to effectively disentangle latent factors. We demonstrate that β-VAE, with β<1, can not only attain good disentanglement but also significantly improve reconstruction accuracy via dynamic control. The paper \textit{removes the inherent trade-off} between reconstruction accuracy and disentanglement for β-VAE. Existing methods, such as β-VAE and FactorVAE, assign a large weight to the KL-divergence term in the objective function, leading to high reconstruction errors for the sake of better disentanglement. To mitigate this problem, a ControlVAE has recently been developed that dynamically tunes the KL-divergence weight in an attempt to \textit{control the trade-off} to more a favorable point. However, ControlVAE fails to eliminate the conflict between the need for a large β (for disentanglement) and the need for a small β (for smaller reconstruction error). Instead, we propose DynamicVAE that maintains a different β at different stages of training, thereby \textit{decoupling disentanglement and reconstruction accuracy}. In order to evolve the weight, β, along a trajectory that enables such decoupling, DynamicVAE leverages a modified incremental PI (proportional-integral) controller, a variant of proportional-integral-derivative controller (PID) algorithm, and employs a moving average as well as a hybrid annealing method to evolve the value of KL-divergence smoothly in a tightly controlled fashion. We theoretically prove the stability of the proposed approach. Evaluation results on three benchmark datasets demonstrate that DynamicVAE significantly improves the reconstruction accuracy while achieving disentanglement comparable to the best of existing methods. The results verify that our method can separate disentangled representation learning and reconstruction, removing the inherent tension between the two.

Shape complexity estimation using VAE

Hidden Talents of the Variational Autoencoder

How to train your VAE

Latent Space Representation for Shape Analysis and Learning

DynamicVAE: Decoupling Reconstruction Error and Disentangled Representation Learning

3D Shape Retrieval Based on Laplace Operator and Joint Bayesian Model

Shape generation via learning an adaptive multimodal prior

Hidden in Plain Sight: Evaluating Abstract Shape Recognition in Vision-Language Models

ED-VAE: Entropy Decomposition of ELBO in Variational Autoencoders

Abstracted Shapes as Tokens -- A Generalizable and Interpretable Model for Time-series Classification

Towards Model-Agnostic Posterior Approximation for Fast and Accurate Variational Autoencoders

Adversarial robustness of VAEs through the lens of local geometry

Generated Loss, Augmented Training, and Multiscale VAE

Improving Variational Autoencoders with Density Gap-based Regularization

Learning Dynamics in Linear VAE: Posterior Collapse Threshold, Superfluous Latent Space Pitfalls, and Speedup with KL Annealing

VAEs in the Presence of Missing Data

Optimizing Hierarchical Image VAEs for Sample Quality

SC-VAE: Sparse Coding-based Variational Autoencoder with Learned ISTA

From abstract items to latent spaces to observed data and back: Compositional Variational Auto-Encoder

Connections with Robust PCA and the Role of Emergent Sparsity in Variational Autoencoder Models

Epanechnikov Variational Autoencoder