Abstract:High-fidelity reconstruction of dynamic fluids from sparse multiview RGB videos remains a formidable challenge, due to the complexity of the underlying physics as well as the severe occlusion and complex lighting in the captured data. Existing solutions either assume knowledge of obstacles and lighting, or only focus on simple fluid scenes without obstacles or complex lighting, and thus are unsuitable for real-world scenes with unknown lighting conditions or arbitrary obstacles. We present the first method to reconstruct dynamic fluid phenomena by leveraging the governing physics (ie, Navier -Stokes equations) in an end-to-end optimization from a mere set of sparse video frames without taking lighting conditions, geometry information, or boundary conditions as input. Our method provides a continuous spatio-temporal scene representation using neural networks as the ansatz of density and velocity solution functions for fluids as well as the radiance field for static objects. With a hybrid architecture that separates static and dynamic contents apart, fluid interactions with static obstacles are reconstructed for the first time without additional geometry input or human labeling. By augmenting time-varying neural radiance fields with physics-informed deep learning, our method benefits from the supervision of images and physical priors. Our progressively growing model with regularization further disentangles the density-color ambiguity in the radiance field, which allows for a more robust optimization from the given input of sparse views. A pretrained density-to-velocity fluid model is leveraged in addition as the data prior to avoid suboptimal velocity solutions which underestimate vorticity but trivially fulfill physical equations. Our method exhibits high-quality results with relaxed constraints and strong flexibility on a representative set of synthetic and real flow captures. Code and sample tests are at https://people.mpi-inf.mpg.de/~mchu/projects/PI-NeRF/.

Learning fluid physics from highly turbulent data using sparse physics-informed discovery of empirical relations (SPIDER)

Learning fluid physics from highly turbulent data using sparse physics-informed discovery of empirical relations (SPIDER)

Fast Uncertainty Quantification by Sparse Data Learning from Multiphysics Systems

Physics-Based Deep Learning for Flow Problems

Physics informed machine learning with Smoothed Particle Hydrodynamics: Hierarchy of reduced Lagrangian models of turbulence

Physics-informed learning of governing equations from scarce data

Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensors

A data-driven smoothed particle hydrodynamics method for fluids

Learning the Latent dynamics of Fluid flows from High-Fidelity Numerical Simulations using Parsimonious Diffusion Maps

Data discovery of low dimensional fluid dynamics of turbulent flows

Inferring Parameters and Reconstruction of Two-dimensional Turbulent Flows with Physics-informed Neural Networks

Interplay of Sensor Quantity, Placement and System Dimensionality on Energy Sparse Reconstruction of Fluid Flows

Robust learning from noisy, incomplete, high-dimensional experimental data via physically constrained symbolic regression

Feel the Force: From Local Surface Pressure Measurement to Flow Reconstruction in Fluid-Structure Interaction

Physics informed neural fields for smoke reconstruction with sparse data

Validation and parameterization of a novel physics-constrained neural dynamics model applied to turbulent fluid flow

Physics-Informed and Data-Driven Discovery of Governing Equations for Complex Phenomena in Heterogeneous Media

Machine Discovery of Partial Differential Equations from Spatiotemporal Data: A Sparse Bayesian Learning Framework.

Feel the force: From local surface pressure measurement to flow reconstruction in fluid–structure interaction

JAX-SPH: A Differentiable Smoothed Particle Hydrodynamics Framework