Abstract:Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables and these models use a nonlinear function (generator) to map latent samples into the data space. On the other hand, the nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. This weak projection, however, can be addressed by a Riemannian metric, and we show that geodesics computation and accurate interpolations between data samples on the Riemannian manifold can substantially improve the performance of deep generative models. In this paper, a Variational spatial-Transformer AutoEncoder (VTAE) is proposed to minimize geodesics on a Riemannian manifold and improve representation learning. In particular, we carefully design the variational autoencoder with an encoded spatial-Transformer to explicitly expand the latent variable model to data on a Riemannian manifold, and obtain global context modelling. Moreover, to have smooth and plausible interpolations while traversing between two different objects' latent representations, we propose a geodesic interpolation network different from the existing models that use linear interpolation with inferior performance. Experiments on benchmarks show that our proposed model can improve predictive accuracy and versatility over a range of computer vision tasks, including image interpolations, and reconstructions.

Fuse It More Deeply! A Variational Transformer with Layer-Wise Latent Variable Inference for Text Generation

A Transformer-Based Hierarchical Variational AutoEncoder Combined Hidden Markov Model for Long Text Generation

Recurrence Boosts Diversity! Revisiting Recurrent Latent Variable in Transformer-Based Variational AutoEncoder for Diverse Text Generation

VOLTA: Improving Generative Diversity by Variational Mutual Information Maximizing Autoencoder

Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoders

AdaVAE: Exploring Adaptive GPT-2s in Variational Auto-Encoders for Language Modeling

Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck

Deep Transformers with Latent Depth

Improved Variational Autoencoders for Text Modeling using Dilated Convolutions

A Temporal Variational Model for Story Generation

Latte: Latent Diffusion Transformer for Video Generation

A Hierarchical Latent Structure for Variational Conversation Modeling

A Bilingual Generative Transformer for Semantic Sentence Embedding

DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer

VTAE: Variational Transformer Autoencoder with Manifolds Learning

LlaMaVAE: Guiding Large Language Model Generation via Continuous Latent Sentence Spaces

Dirichlet Latent Variable Hierarchical Recurrent Encoder-Decoder in Dialogue Generation.

GIVT: Generative Infinite-Vocabulary Transformers

T-CVAE: Transformer-Based Conditioned Variational Autoencoder for Story Completion

Improving Semantic Control in Discrete Latent Spaces with Transformer Quantized Variational Autoencoders

Latent-Glat: Glancing at Latent Variables for Parallel Text Generation