Abstract:We present a deep-learning Variational Encoder-Decoder (VED) framework for learning data-driven low-dimensional representations of the relationship between high-dimensional parameters of a physical system and the system's high-dimensional observable response. The framework consists of two deep learning-based probabilistic transformations: An encoder mapping parameters to latent codes and a decoder mapping latent codes to the observable response. The hyperparameters of these transformations are identified by maximizing a variational lower bound on the log-conditional distribution of the observable response given parameters. To promote the disentanglement of latent codes, we equip this variational loss with a penalty on the off-diagonal entries of the aggregate distribution covariance of codes. This regularization penalty encourages the pushforward of a standard Gaussian distribution of latent codes to approximate the marginal distribution of the observable response. Using the proposed framework we successfully model the hydraulic pressure response at observation wells of a groundwater flow model as a function of its discrete log-hydraulic transmissivity field. Compared to the canonical correlation analysis encoding, the VED model achieves a lower-dimensional latent representation, with as low as $r = 50$ latent dimensions without a significant loss of reconstruction accuracy. We explore the impact of regularization on model performance, finding that KL-divergence and covariance regularization improve feature disentanglement in latent space while maintaining reconstruction accuracy. Furthermore, we evaluate the generative capabilities of the regularized model by decoding random Gaussian noise, revealing that tuning both $\beta$ and $\lambda$ parameters enhances the quality of the generated observable response data.

Latent Variable Modelling Using Variational Autoencoders: A survey

Hidden Talents of the Variational Autoencoder

Different Latent Variables Learning In Variational Autoencoder

An Introduction to Variational Autoencoders

From the Expectation Maximisation Algorithm to Autoencoded Variational Bayes

Asymmetric Variational Autoencoders

Factor Analysis, Probabilistic Principal Component Analysis, Variational Inference, and Variational Autoencoder: Tutorial and Survey

Exploring the Potential of Variational Autoencoders for Modeling Nonlinear Relationships in Psychological Data

A Survey on Variational Autoencoders in Recommender Systems

Latent Space Characterization of Autoencoder Variants

Variational Classification

Nonlinear Probabilistic Latent Variable Regression Models for Soft Sensor Application: from Shallow to Deep Structure

Auto-Encoding Variational Bayes

Using VAEs to Learn Latent Variables: Observations on Applications in cryo-EM

Variational Autoencoders for Efficient Simulation-Based Inference

A Survey on Variational Autoencoders from a Green AI Perspective

Exploring the Latent Space of Autoencoders with Interventional Assays

Variational Encoder-Decoders for Learning Latent Representations of Physical Systems

Three Variations on Variational Autoencoders

Learning Summary Statistics for Bayesian Inference with Autoencoders

Prior latent distribution comparison for the RNN Variational Autoencoder in low-resource language modeling