Abstract:High-fidelity reconstruction of dynamic fluids from sparse multiview RGB videos remains a formidable challenge, due to the complexity of the underlying physics as well as the severe occlusion and complex lighting in the captured data. Existing solutions either assume knowledge of obstacles and lighting, or only focus on simple fluid scenes without obstacles or complex lighting, and thus are unsuitable for real-world scenes with unknown lighting conditions or arbitrary obstacles. We present the first method to reconstruct dynamic fluid phenomena by leveraging the governing physics (ie, Navier -Stokes equations) in an end-to-end optimization from a mere set of sparse video frames without taking lighting conditions, geometry information, or boundary conditions as input. Our method provides a continuous spatio-temporal scene representation using neural networks as the ansatz of density and velocity solution functions for fluids as well as the radiance field for static objects. With a hybrid architecture that separates static and dynamic contents apart, fluid interactions with static obstacles are reconstructed for the first time without additional geometry input or human labeling. By augmenting time-varying neural radiance fields with physics-informed deep learning, our method benefits from the supervision of images and physical priors. Our progressively growing model with regularization further disentangles the density-color ambiguity in the radiance field, which allows for a more robust optimization from the given input of sparse views. A pretrained density-to-velocity fluid model is leveraged in addition as the data prior to avoid suboptimal velocity solutions which underestimate vorticity but trivially fulfill physical equations. Our method exhibits high-quality results with relaxed constraints and strong flexibility on a representative set of synthetic and real flow captures. Code and sample tests are at https://people.mpi-inf.mpg.de/~mchu/projects/PI-NeRF/.

Data recovery in computational fluid dynamics through deep image priors

Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning

Probabilistic neural networks for fluid flow surrogate modeling and data recovery

Physics-Based Deep Learning for Flow Problems

Inpainting Computational Fluid Dynamics with Deep Learning

PiRD: Physics-informed Residual Diffusion for Flow Field Reconstruction

Shallow Neural Networks for Fluid Flow Reconstruction with Limited Sensors

A Fluid Flow Data Set for Machine Learning and its Application to Neural Flow Map Interpolation

Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flows

Graph Neural Networks for Aerodynamic Flow Reconstruction from Sparse Sensing

A practical approach to flow field reconstruction with sparse or incomplete data through physics informed neural network

Reconstructing turbulent velocity information for arbitrarily gappy flow fields using the deep convolutional neural network

PINNs4Drops: Convolutional feature-enhanced physics-informed neural networks for reconstructing two-phase flows

Deep Learning combined with singular value decomposition to reconstruct databases in fluid dynamics

Unsupervised Denoising and Super-Resolution of Vascular Flow Data by Physics-informed Machine Learning

Reconstructing unsteady flows from sparse, noisy measurements with a physics-constrained convolutional neural network

Physics informed neural fields for smoke reconstruction with sparse data

Physics-informed deep-learning applications to experimental fluid mechanics

Machine learning accelerated computational fluid dynamics

DeepCFD: Efficient Steady-State Laminar Flow Approximation with Deep Convolutional Neural Networks

Learning Incompressible Fluid Dynamics from Scratch -- Towards Fast, Differentiable Fluid Models that Generalize