Abstract:High-fidelity reconstruction of dynamic fluids from sparse multiview RGB videos remains a formidable challenge, due to the complexity of the underlying physics as well as the severe occlusion and complex lighting in the captured data. Existing solutions either assume knowledge of obstacles and lighting, or only focus on simple fluid scenes without obstacles or complex lighting, and thus are unsuitable for real-world scenes with unknown lighting conditions or arbitrary obstacles. We present the first method to reconstruct dynamic fluid phenomena by leveraging the governing physics (ie, Navier -Stokes equations) in an end-to-end optimization from a mere set of sparse video frames without taking lighting conditions, geometry information, or boundary conditions as input. Our method provides a continuous spatio-temporal scene representation using neural networks as the ansatz of density and velocity solution functions for fluids as well as the radiance field for static objects. With a hybrid architecture that separates static and dynamic contents apart, fluid interactions with static obstacles are reconstructed for the first time without additional geometry input or human labeling. By augmenting time-varying neural radiance fields with physics-informed deep learning, our method benefits from the supervision of images and physical priors. Our progressively growing model with regularization further disentangles the density-color ambiguity in the radiance field, which allows for a more robust optimization from the given input of sparse views. A pretrained density-to-velocity fluid model is leveraged in addition as the data prior to avoid suboptimal velocity solutions which underestimate vorticity but trivially fulfill physical equations. Our method exhibits high-quality results with relaxed constraints and strong flexibility on a representative set of synthetic and real flow captures. Code and sample tests are at https://people.mpi-inf.mpg.de/~mchu/projects/PI-NeRF/.

Global Physical Prior Based Fluid Reconstruction for VR/AR

Laplacian Projection Based Global Physical Prior Smoke Reconstruction

Parallel Realistic Visualization of Particle‐based Fluid

Inferring Fluid Dynamics via Inverse Rendering

Visual Perception of Fluid Viscosity: Toward Realistic Fluid Simulation.

Physics informed neural fields for smoke reconstruction with sparse data

A Generalized Variational Level Set Method Without Frequent Reinitialization for Simulations of Gas-Liquid Flows.

A Hybrid Modeling Method for Dynamic Liquid Simulation

Non-Newtonian fluid simulation and reconstruction from monocular videos

Real-time High-Quality Surface Rendering for Large Scale Particle-Based Fluids

A Generalized Coupled Level Set/volume-of-fluid/ghost Fluid Method for Detailed Simulation of Gas-Liquid Flows.

View-Dependent Multiscale Fluid Simulation

FluidPlaying: Efficient Adaptive Simulation for Highly Dynamic Fluid

PhyRecon: Physically Plausible Neural Scene Reconstruction

3D-IntPhys: Towards More Generalized 3D-grounded Visual Intuitive Physics under Challenging Scenes

PIV-Based 3D Fluid Flow Reconstruction Using Light Field Camera

Latent Intuitive Physics: Learning to Transfer Hidden Physics from A 3D Video

PiRD: Physics-informed Residual Diffusion for Flow Field Reconstruction

Physics-based fluid simulation in computer graphics: Survey, research trends, and challenges

Bayesian Reconstruction of 3D Particle Positions in High-Seeding Density Flows

Physics perception in sloshing scenes with guaranteed thermodynamic consistency