Abstract:We develop Riemannian approaches to variational autoencoders (VAEs) for PDE-type ambient data with regularizing geometric latent dynamics, which we refer to as VAE-DLM, or VAEs with dynamical latent manifolds. We redevelop the VAE framework such that manifold geometries, subject to our geometric flow, embedded in Euclidean space are learned in the intermediary latent space developed by encoders and decoders. By tailoring the geometric flow in which the latent space evolves, we induce latent geometric properties of our choosing, which are reflected in empirical performance. We reformulate the traditional evidence lower bound (ELBO) loss with a considerate choice of prior. We develop a linear geometric flow with a steady-state regularizing term. This flow requires only automatic differentiation of one time derivative, and can be solved in moderately high dimensions in a physics-informed approach, allowing more expressive latent representations. We discuss how this flow can be formulated as a gradient flow, and maintains entropy away from metric singularity. This, along with an eigenvalue penalization condition, helps ensure the manifold is sufficiently large in measure, nondegenerate, and a canonical geometry, which contribute to a robust representation. Our methods focus on the modified multi-layer perceptron architecture with tanh activations for the manifold encoder-decoder. We demonstrate, on our datasets of interest, our methods perform at least as well as the traditional VAE, and oftentimes better. Our methods can outperform this and a VAE endowed with our proposed architecture by up to 25% reduction in out-of-distribution (OOD) error and potentially greater. We highlight our method on ambient PDEs whose solutions maintain minimal variation in late times. We provide empirical justification towards how we can improve robust learning for external dynamics with VAEs.

GD-VAEs: Geometric Dynamic Variational Autoencoders for Learning Nonlinear Dynamics and Dimension Reductions

Variational autoencoders with latent high-dimensional steady geometric flows for dynamics

Variational Encoder-Decoders for Learning Latent Representations of Physical Systems

Neighborhood Geometric Structure-Preserving Variational Autoencoder for Smooth and Bounded Data Sources

Supervised Nonlinear Dynamic System for Soft Sensor Application Aided by Variational Auto-Encoder

$Γ$-VAE: Curvature regularized variational autoencoders for uncovering emergent low dimensional geometric structure in high dimensional data

$Φ$-DVAE: Physics-Informed Dynamical Variational Autoencoders for Unstructured Data Assimilation

Symplectic encoders for physics-constrained variational dynamics inference

D-VAE: A Variational Autoencoder for Directed Acyclic Graphs

VTAE: Variational Transformer Autoencoder with Manifolds Learning

Dynamic Joint Variational Graph Autoencoders

Learning Discrete Structured Variational Auto-Encoder using Natural Evolution Strategies

eXponential FAmily Dynamical Systems (XFADS): Large-scale nonlinear Gaussian state-space modeling

Hidden Talents of the Variational Autoencoder

Multi-modal Gaussian Process Variational Autoencoders for Neural and Behavioral Data

Disentangling Generative Factors of Physical Fields Using Variational Autoencoders

Learning Latent Dynamics via Invariant Decomposition and (Spatio-)Temporal Transformers

Physics-enhanced Gaussian Process Variational Autoencoder

Dynamical Variational Autoencoders: A Comprehensive Review

Physics and Chemistry from Parsimonious Representations: Image Analysis via Invariant Variational Autoencoders

Nonparametric Variational Auto-encoders for Hierarchical Representation Learning.