Abstract:Variational Auto-Encoders (VAEs) have been widely applied for learning compact, low-dimensional latent representations of high-dimensional data. When the correlation structure among data points is available, previous work proposed Correlated Variational Auto-Encoders (CVAEs), which employ a structured mixture model as prior and a structured variational posterior for each mixture component to enforce that the learned latent representations follow the same correlation structure. However, as we demonstrate in this work, such a choice cannot guarantee that CVAEs capture all the correlations. Furthermore, it prevents us from obtaining a tractable joint and marginal variational distribution. To address these issues, we propose Adaptive Correlated Variational Auto-Encoders (ACVAEs), which apply an adaptive prior distribution that can be adjusted during training and can learn a tractable joint variational distribution. Its tractable form also enables further refinement with belief propagation. Experimental results on link prediction and hierarchical clustering show that ACVAEs significantly outperform CVAEs among other benchmarks.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to effectively capture the correlations between data points in the variational auto - encoder (VAE), especially when there are known structured correlations between data points. Specifically, the paper points out that although the traditional CVAE (Correlated Variational Auto - Encoders) takes into account the correlations between data points, it has the following limitations: 1. **Inability to fully capture all correlations**: The prior distribution of CVAE is a uniform mixture based on the maximum acyclic subgraph, which may lead to some subgraphs not being able to capture the correlations between data points well. 2. **Difficulty in calculating joint and marginal variational distributions**: The structured prior of CVAE makes its joint and marginal variational distributions non - analytic, which affects the performance of downstream tasks. 3. **High pre - processing time complexity**: CVAE requires a pre - processing step with a time complexity of O(|V|^3) to calculate edge weights, limiting its application on large - scale data sets. To solve these problems, the paper proposes ACVAE (Adaptive Correlated Variational Auto - Encoders). By introducing an adaptive non - uniform mixture prior distribution and allowing the weights of these prior distributions to be adjusted during the training process, the above problems are overcome. Specific improvements include: - **Non - uniform mixture prior**: Use a non - uniform mixture prior distribution instead of a uniform mixture prior, enabling the model to better adapt to the correlations of different subgraphs. - **Analytic joint variational distribution**: By adaptively adjusting the prior distribution, ensure that an analytic joint variational distribution is finally obtained, so that accurate marginal inference can be carried out. - **Avoid high - time - complexity pre - processing**: ACVAE does not require the high - time - complexity pre - processing step required by CVAE, enabling it to be applied to larger - scale data sets. Experimental results show that ACVAE is significantly superior to CVAE and other benchmark methods in tasks such as link prediction and hierarchical clustering, especially when combined with belief propagation for accurate marginal inference. ### Summary The main contribution of this paper is to propose a new adaptive correlated variational auto - encoder (ACVAE), which solves the limitations of the traditional CVAE in capturing data point correlations and computational efficiency, thereby improving the performance of the model in multiple tasks.

Learning Correlated Latent Representations with Adaptive Priors

Correlating Variational Autoencoders Natively For Multi-View Imputation

On the Necessity and Effectiveness of Learning the Prior of Variational Auto-Encoder.

Learning conditional variational autoencoders with missing covariates

Enhancing VAEs for Collaborative Filtering: Flexible Priors & Gating Mechanisms

Tychite, Na 6 Mg 2 (SO 4 )(CO 3 ) 4 : structure analysi

Towards Model-Agnostic Posterior Approximation for Fast and Accurate Variational Autoencoders

Associative Compression Networks for Representation Learning

Learning Audio-Visual Correlations from Variational Cross-Modal Generation

Hidden Talents of the Variational Autoencoder

C$^2$VAE: Gaussian Copula-based VAE Differing Disentangled from Coupled Representations with Contrastive Posterior

tvGP-VAE: Tensor-variate Gaussian Process Prior Variational Autoencoder

Connections with Robust PCA and the Role of Emergent Sparsity in Variational Autoencoder Models

Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks

Sparse-Coding Variational Autoencoders

Improving Variational Autoencoders with Density Gap-based Regularization

Sparse-Coding Variational Auto-Encoders

D-VAE: A Variational Autoencoder for Directed Acyclic Graphs

VAEGAN: A Collaborative Filtering Framework based on Adversarial Variational Autoencoders

Neighbor Embedding Variational Autoencoder

How to train your VAE