Abstract:The occlusion problem remains a crucial challenge in optical flow estimation (OFE). Despite the recent significant progress brought about by deep learning, most existing deep learning OFE methods still struggle to handle occlusions; in particular, those based on two frames cannot correctly handle occlusions because occluded regions have no visual correspondences. However, there is still hope in multi-frame settings, which can potentially mitigate the occlusion issue in OFE. Unfortunately, multi-frame OFE (MOFE) remains underexplored, and the limited studies on it are mainly specially designed for pyramid backbones or else obtain the aligned previous frame's features, such as correlation volume and optical flow, through time-consuming backward flow calculation or non-differentiable forward warping transformation. This study proposes an efficient MOFE framework named SplatFlow to address these shortcomings. SplatFlow introduces the differentiable splatting transformation to align the previous frame's motion feature and designs a Final-to-All embedding method to input the aligned motion feature into the current frame's estimation, thus remodeling the existing two-frame backbones. The proposed SplatFlow is efficient yet more accurate, as it can handle occlusions properly. Extensive experimental evaluations show that SplatFlow substantially outperforms all published methods on the KITTI2015 and Sintel benchmarks. Especially on the Sintel benchmark, SplatFlow achieves errors of 1.12 (clean pass) and 2.07 (final pass), with surprisingly significant 19.4% and 16.2% error reductions, respectively, from the previous best results submitted. The code for SplatFlow is available at https://github.com/wwsource/SplatFlow.

Gcpof: A Novel Optical Flow Estimation Algorithm Leveraged By Sparse Ground Control Points

A Dense Optical Flow Registration Algorithm Based on Deep Learning

Optical Flow Dataset and Benchmark for Visual Crowd Analysis

Efficient Coarse-to-Fine Patch Match for Large Displacement Optical Flow.

An Efficient Gyro-Aided Optical Flow Estimation in Fast Rotations with Auto-Calibration

Context-adaptive Matching for Optical Flow

Large Displacement Optical Flow Estimation Based on Robust Interpolation of Sparse Correspondences

SplatFlow: Learning Multi-frame Optical Flow via Splatting

Explicit Motion Disentangling for Efficient Optical Flow Estimation

Efficient Segmentation-Based PatchMatch for Large Displacement Optical Flow Estimation

A Robust Optical Flow Computation

Robust Optical Flow Computation: A Higher-Order Differential Approach

Optical Flow Method for Motion Details Estimation

Geodesic‐based Probability Propagation for Efficient Optical Flow

Fast Optical Flow Estimation Without Parallel Architectures.

NS-SOF: A Non-feature Matching Approach for Sparse Optical Flow

Fast Optical Flow Estimation Based on the Split Bregman Method

Robust computation of optical flow field with large motion

DeGraF-Flow: Extending DeGraF Features for accurate and efficient sparse-to-dense optical flow estimation

A Filtering-Based Framework for Optical Flow Estimation