Abstract:We develop a new Bayesian framework based on deep neural networks to be able to extrapolate in space-time using historical data and to quantify uncertainties arising from both noisy and gappy data in physical problems. Specifically, the proposed approach has two stages: (1) prior learning and (2) posterior estimation. At the first stage, we employ the physics-informed Generative Adversarial Networks (PI-GAN) to learn a functional prior either from a prescribed function distribution, e.g., Gaussian process, or from historical data and physics. At the second stage, we employ the Hamiltonian Monte Carlo (HMC) method to estimate the posterior in the latent space of PI-GANs. In addition, we use two different approaches to encode the physics: (1) automatic differentiation, used in the physics-informed neural networks (PINNs) for scenarios with explicitly known partial differential equations (PDEs), and (2) operator regression using the deep operator network (DeepONet) for PDE-agnostic scenarios. We then test the proposed method for (1) meta-learning for one-dimensional regression, and forward/inverse PDE problems (combined with PINNs); (2) PDE-agnostic physical problems (combined with DeepONet), e.g., fractional diffusion as well as saturated stochastic (100-dimensional) flows in heterogeneous porous media; and (3) spatial-temporal regression problems, i.e., inference of a marine riser displacement field using experimental data from the Norwegian Deepwater Programme (NDP). The results demonstrate that the proposed approach can provide accurate predictions as well as uncertainty quantification given very limited scattered and noisy data, since historical data could be available to provide informative priors. In , the proposed method is capable of learning flexible functional priors, e.g., both Gaussian and non-Gaussian process, and can be readily extended to big data problems by enabling mini-batch training using stochastic HMC or normalizing flows since the latent space is generally characterized as low dimensional.

Adaptive Latent Space Tuning for Non-Stationary Distributions

Adaptive Machine Learning for Time-Varying Systems: Low Dimensional Latent Space Tuning

STONet: A Neural-Operator-Driven Spatio-temporal Network

Towards latent space evolution of spatiotemporal dynamics of six-dimensional phase space of charged particle beams

CBOL-Tuner: Classifier-pruned Bayesian optimization to explore temporally structured latent spaces for particle accelerator tuning

Learning nonlinear operators in latent spaces for real-time predictions of complex dynamics in physical systems

Learning in latent spaces improves the predictive accuracy of deep neural operators

Latent Dynamics Networks (LDNets): learning the intrinsic dynamics of spatio-temporal processes

Learning High-Dimensional Distributions with Latent Neural Fokker-Planck Kernels

Continual Learning with Pretrained Backbones by Tuning in the Input Space

Learning the intrinsic dynamics of spatio-temporal processes through Latent Dynamics Networks

Adaptive Machine Learning for Time-Varying Systems: Towards 6D Phase Space Diagnostics of Short Intense Charged Particle Beams

Learning Latent Dynamics via Invariant Decomposition and (Spatio-)Temporal Transformers

Time-inversion of spatiotemporal beam dynamics using uncertainty-aware latent evolution reversal

Towards Kinetic Manipulation of the Latent Space

Latent Space Dynamics Learning for Stiff Collisional-radiative Models

Learning Controllable Adaptive Simulation for Multi-resolution Physics

Adaptive Learning of the Latent Space of Wasserstein Generative Adversarial Networks

Learning functional priors and posteriors from data and physics

Adapt & Align: Continual Learning with Generative Models Latent Space Alignment

End-To-End Latent Variational Diffusion Models for Inverse Problems in High Energy Physics