Abstract:Large displacement optical flow is an integral part of many computer vision tasks. Variational optical flow techniques based on a coarse-to-fine scheme interpolate sparse matches and locally optimize an energy model conditioned on colour, gradient and smoothness, making them sensitive to noise in the sparse matches, deformations, and arbitrarily large displacements. This paper addresses this problem and presents HybridFlow, a variational motion estimation framework for large displacements and deformations. A multi-scale hybrid matching approach is performed on the image pairs. Coarse-scale clusters formed by classifying pixels according to their feature descriptors are matched using the clusters' context descriptors. We apply a multi-scale graph matching on the finer-scale superpixels contained within each matched pair of coarse-scale clusters. Small clusters that cannot be further subdivided are matched using localized feature matching. Together, these initial matches form the flow, which is propagated by an edge-preserving interpolation and variational refinement. Our approach does not require training and is robust to substantial displacements and rigid and non-rigid transformations due to motion in the scene, making it ideal for large-scale imagery such as Wide-Area Motion Imagery (WAMI). More notably, HybridFlow works on directed graphs of arbitrary topology representing perceptual groups, which improves motion estimation in the presence of significant deformations. We demonstrate HybridFlow's superior performance to state-of-the-art variational techniques on two benchmark datasets and report comparable results with state-of-the-art deep-learning-based techniques.

Scale-Space SIFT flow

Prior Information Constrained SIFT Matching Algorithm for Visual Simultaneous Localization and Mapping

SIFT Flow for Large-Displacement Object Tracking

SIFT Implementation Based on Parallel Computation

Scale-flow: Estimating 3D Motion from Video

Progressive Large Scale-Invariant Image Matching in Scale Space.

Fast Scene Matching Method Based on Scale Invariant Feature Transform

ScaleFlow++: Robust and Accurate Estimation of 3D Motion from Video

SIFT algorithm based on the large-format image block

STDC‐Flow: Large Displacement Flow Field Estimation Using Similarity Transformation‐based Dense Correspondence

DELFlow: Dense Efficient Learning of Scene Flow for Large-Scale Point Clouds

A Comparative Study of SIFT and Its Variants

Multi-Scale Generalized Plane Match for Optical Flow

Motion Estimation for Large Displacements and Deformations

A multi-scale optical flow algorithm based on oriented smoothness constraint optimization

Scale-space Texture Description on SIFT-like Textons

Modified Scale Invariant Feature Transform in omnidirectional images

Contourlet-SIFT Feature Matching Algorithm

Infrared Weak Target Detection and Tracking Based on SIFT Flow

Context-adaptive Matching for Optical Flow