Abstract:We present a deep-learning Variational Encoder-Decoder (VED) framework for learning data-driven low-dimensional representations of the relationship between high-dimensional parameters of a physical system and the system's high-dimensional observable response. The framework consists of two deep learning-based probabilistic transformations: An encoder mapping parameters to latent codes and a decoder mapping latent codes to the observable response. The hyperparameters of these transformations are identified by maximizing a variational lower bound on the log-conditional distribution of the observable response given parameters. To promote the disentanglement of latent codes, we equip this variational loss with a penalty on the off-diagonal entries of the aggregate distribution covariance of codes. This regularization penalty encourages the pushforward of a standard Gaussian distribution of latent codes to approximate the marginal distribution of the observable response. Using the proposed framework we successfully model the hydraulic pressure response at observation wells of a groundwater flow model as a function of its discrete log-hydraulic transmissivity field. Compared to the canonical correlation analysis encoding, the VED model achieves a lower-dimensional latent representation, with as low as $r = 50$ latent dimensions without a significant loss of reconstruction accuracy. We explore the impact of regularization on model performance, finding that KL-divergence and covariance regularization improve feature disentanglement in latent space while maintaining reconstruction accuracy. Furthermore, we evaluate the generative capabilities of the regularized model by decoding random Gaussian noise, revealing that tuning both $\beta$ and $\lambda$ parameters enhances the quality of the generated observable response data.

Use of Student's t-Distribution for the Latent Layer in a Coupled Variational Autoencoder

$t^3$-Variational Autoencoder: Learning Heavy-tailed Data with Student's t and Power Divergence

Generated Loss and Augmented Training of MNIST VAE

Hidden Talents of the Variational Autoencoder

Protect Before Generate: Error Correcting Codes within Discrete Deep Generative Models

Image generation via latent space learning using improved combination

Variational Classification

Adaptive Learning of the Latent Space of Wasserstein Generative Adversarial Networks

Langevin Autoencoders for Learning Deep Latent Variable Models

Cooperation in the Latent Space: The Benefits of Adding Mixture Components in Variational Autoencoders

Variational Encoder-Decoders for Learning Latent Representations of Physical Systems

An adaptive dimension reduction algorithm for latent variables of variational autoencoder.

Learning Multi-layer Latent Variable Model via Variational Optimization of Short Run MCMC for Approximate Inference

Uniform Transformation: Refining Latent Representation in Variational Autoencoders

Stacked Wasserstein Autoencoder

Variance Loss in Variational Autoencoders

Ladder Variational Autoencoders

Multilinear Latent Conditioning for Generating Unseen Attribute Combinations

Diffusion Variational Autoencoders

Different Latent Variables Learning In Variational Autoencoder

Adaptive Density Estimation for Generative Models