Abstract:The accuracy of learning-based optical flow estimation models heavily relies on the realism of the training datasets. Current approaches for generating such datasets either employ synthetic data or generate images with limited realism. However, the domain gap of these data with real-world scenes constrains the generalization of the trained model to real-world applications. To address this issue, we investigate generating realistic optical flow datasets from real-world images. Firstly, to generate highly realistic new images, we construct a layered depth representation, known as multiplane images (MPI), from single-view images. This allows us to generate novel view images that are highly realistic. To generate optical flow maps that correspond accurately to the new image, we calculate the optical flows of each plane using the camera matrix and plane depths. We then project these layered optical flows into the output optical flow map with volume rendering. Secondly, to ensure the realism of motion, we present an independent object motion module that can separate the camera and dynamic object motion in MPI. This module addresses the deficiency in MPI-based single-view methods, where optical flow is generated only by camera motion and does not account for any object movement. We additionally devise a depth-aware inpainting module to merge new images with dynamic objects and address unnatural motion occlusions. We show the superior performance of our method through extensive experiments on real-world datasets. Moreover, our approach achieves state-of-the-art performance in both unsupervised and supervised training of learning-based models. The code will be made publicly available at: \url{<a class="link-external link-https" href="https://github.com/Sharpiless/MPI-Flow" rel="external noopener nofollow">this https URL</a>}.

Deep-learning Optical Flow Outperforms PIV in Obtaining Velocity Fields from Active Nematics

Deep-learning Optical Flow for Measuring Velocity Fields from Experimental Data

Particle Image Velocimetry Based on a Deep Learning Motion Estimator.

A Velocity Decomposition-Based 3D Optical Flow Method for Accurate Tomo-PIV Measurement

A Dense Optical Flow Registration Algorithm Based on Deep Learning

Particle image velocimetry based on a deep neural network

Deep-PIV: a new framework of PIV using deep learning techniques

DeepPTV: Particle Tracking Velocimetry for Complex Flow Motion via Deep Neural Networks

Recurrent Graph Optimal Transport for Learning 3D Flow Motion in Particle Tracking

Deep learning-based optical flow analysis of two-dimensional Rayleigh scattering imaging of high-speed flows

HMAFlow: Learning More Accurate Optical Flow via Hierarchical Motion Field Alignment

Generalization of deep recurrent optical flow estimation for particle-image velocimetry data

Real-time and On-site Aerodynamics using Stereoscopic PIV and Deep Optical Flow Learning

Neural optical flow for planar and stereo PIV

Challenges of deep unsupervised optical flow estimation for particle-image velocimetry data

An Unsupervised Deep Learning Model for Dense Velocity Field Reconstruction in Particle Image Velocimetry (PIV) Measurements

SplatFlow: Learning Multi-frame Optical Flow via Splatting

MPI-Flow: Learning Realistic Optical Flow with Multiplane Images

Towards High Resolution Real-Time Optical Flow Particle Image Velocimetry

Survey on Unsupervised Learning Methods for Optical Flow Estimation