Abstract:Large displacement optical flow is an integral part of many computer vision tasks. Variational optical flow techniques based on a coarse-to-fine scheme interpolate sparse matches and locally optimize an energy model conditioned on colour, gradient and smoothness, making them sensitive to noise in the sparse matches, deformations, and arbitrarily large displacements. This paper addresses this problem and presents HybridFlow, a variational motion estimation framework for large displacements and deformations. A multi-scale hybrid matching approach is performed on the image pairs. Coarse-scale clusters formed by classifying pixels according to their feature descriptors are matched using the clusters' context descriptors. We apply a multi-scale graph matching on the finer-scale superpixels contained within each matched pair of coarse-scale clusters. Small clusters that cannot be further subdivided are matched using localized feature matching. Together, these initial matches form the flow, which is propagated by an edge-preserving interpolation and variational refinement. Our approach does not require training and is robust to substantial displacements and rigid and non-rigid transformations due to motion in the scene, making it ideal for large-scale imagery such as Wide-Area Motion Imagery (WAMI). More notably, HybridFlow works on directed graphs of arbitrary topology representing perceptual groups, which improves motion estimation in the presence of significant deformations. We demonstrate HybridFlow's superior performance to state-of-the-art variational techniques on two benchmark datasets and report comparable results with state-of-the-art deep-learning-based techniques.

Efficient Dense Optical Flow Estimation from Two-Images

A Dense Optical Flow Registration Algorithm Based on Deep Learning

Dense Estimation of Optical Flow Field Within the MPEG-2 Compressed Domain.

A PDE-Based Method for Optical Flow Estimation

Optical Flow Approximation Based Motion Object Extraction for MPEG-2 Video Stream

Flow Fields: Dense Correspondence Fields for Highly Accurate Large Displacement Optical Flow Estimation

Optical Flow Estimation from Layered Nearest Neighbor Flow Fields.

A new algorithm for estimating optic flow for low-level vision systems

Dense Continuous-Time Optical Flow from Events and Frames

Optical Flow Dataset and Benchmark for Visual Crowd Analysis

Multi-Scale Generalized Plane Match for Optical Flow

Robust Optical Flow Computation: A Higher-Order Differential Approach

Motion Estimation for Large Displacements and Deformations

Learning Optical Flow from a Few Matches

NeuFlow v2: High-Efficiency Optical Flow Estimation on Edge Devices

A Dense Optical Flow-Based Feature Matching Approach in Visual Odometry

HMAFlow: Learning More Accurate Optical Flow via Hierarchical Motion Field Alignment

Global Matching with Overlapping Attention for Optical Flow Estimation

A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them

A Robust Optical Flow Computation

Optical Flow Estimation Via Motion Feature Recovery