Abstract:We develop a new Bayesian framework based on deep neural networks to be able to extrapolate in space-time using historical data and to quantify uncertainties arising from both noisy and gappy data in physical problems. Specifically, the proposed approach has two stages: (1) prior learning and (2) posterior estimation. At the first stage, we employ the physics-informed Generative Adversarial Networks (PI-GAN) to learn a functional prior either from a prescribed function distribution, e.g., Gaussian process, or from historical data and physics. At the second stage, we employ the Hamiltonian Monte Carlo (HMC) method to estimate the posterior in the latent space of PI-GANs. In addition, we use two different approaches to encode the physics: (1) automatic differentiation, used in the physics-informed neural networks (PINNs) for scenarios with explicitly known partial differential equations (PDEs), and (2) operator regression using the deep operator network (DeepONet) for PDE-agnostic scenarios. We then test the proposed method for (1) meta-learning for one-dimensional regression, and forward/inverse PDE problems (combined with PINNs); (2) PDE-agnostic physical problems (combined with DeepONet), e.g., fractional diffusion as well as saturated stochastic (100-dimensional) flows in heterogeneous porous media; and (3) spatial-temporal regression problems, i.e., inference of a marine riser displacement field using experimental data from the Norwegian Deepwater Programme (NDP). The results demonstrate that the proposed approach can provide accurate predictions as well as uncertainty quantification given very limited scattered and noisy data, since historical data could be available to provide informative priors. In , the proposed method is capable of learning flexible functional priors, e.g., both Gaussian and non-Gaussian process, and can be readily extended to big data problems by enabling mini-batch training using stochastic HMC or normalizing flows since the latent space is generally characterized as low dimensional.

One-shot backpropagation for multi-step prediction in physics-based system identification -- EXTENDED VERSION

Combining Off-White and Sparse Black Models in Multi-step Physics-based Systems Identification -- EXTENDED VERSION

Stabilizing Backpropagation Through Time to Learn Complex Physics

STEP: extraction of underlying physics with robust machine learning

Learning functional priors and posteriors from data and physics

An extended physics informed neural network for preliminary analysis of parametric optimal control problems

Physics-enhanced Neural Networks in the Small Data Regime

Learning Physics Informed Neural ODEs With Partial Measurements

Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking

Physics Interpretable Shallow-Deep Neural Networks for Physical System Identification with Unobservability.

Extrapolation of Physics-Inspired Deep Networks in Learning Robot Inverse Dynamics

Bidirectional Dynamic Neural Networks with Physical Analyzability

A nonintrusive hybrid neural-physics modeling of incomplete dynamical systems: Lorenz equations

DeepPhysics: a physics aware deep learning framework for real-time simulation

Some of the variables, some of the parameters, some of the times, with some physics known: Identification with partial information

One-Shot Transfer Learning of Physics-Informed Neural Networks

Scale-invariant Learning by Physics Inversion

A Framework Based on Symbolic Regression Coupled with eXtended Physics-Informed Neural Networks for Gray-Box Learning of Equations of Motion from Data

Predicting parametric spatiotemporal dynamics by multi-resolution PDE structure-preserved deep learning

Physics Informed Neural Network Framework for Unsteady Discretized Reduced Order System

Response Estimation and System Identification of Dynamical Systems via Physics-Informed Neural Networks