Abstract:Optical flow estimation is crucial to a variety of vision tasks. Despite substantial recent advancements, achieving real-time on-device optical flow estimation remains a complex challenge. First, an optical flow model must be sufficiently lightweight to meet computation and memory constraints to ensure real-time performance on devices. Second, the necessity for real-time on-device operation imposes constraints that weaken the model's capacity to adequately handle ambiguities in flow estimation, thereby intensifying the difficulty of preserving flow accuracy. This paper introduces two synergistic techniques, Self-Cleaning Iteration (SCI) and Regression Focal Loss (RFL), designed to enhance the capabilities of optical flow models, with a focus on addressing optical flow regression ambiguities. These techniques prove particularly effective in mitigating error propagation, a prevalent issue in optical flow models that employ iterative refinement. Notably, these techniques add negligible to zero overhead in model parameters and inference latency, thereby preserving real-time on-device efficiency. The effectiveness of our proposed SCI and RFL techniques, collectively referred to as SciFlow for brevity, is demonstrated across two distinct lightweight optical flow model architectures in our experiments. Remarkably, SciFlow enables substantial reduction in error metrics (EPE and Fl-all) over the baseline models by up to 6.3% and 10.5% for in-domain scenarios and by up to 6.2% and 13.5% for cross-domain scenarios on the Sintel and KITTI 2015 datasets, respectively.

Variational optic flow on the Sony PlayStation 3

Parallel Implementation of Dense Optical Flow Computation on Many-Core Processor.

An FPGA-Optimized Architecture of Real-time Farneback Optical Flow

NeuFlow v2: High-Efficiency Optical Flow Estimation on Edge Devices

Image-Space Texture-Based Output-Coherent Surface Flow Visualization.

Optical Flow in Mostly Rigid Scenes

Real-time FPGA Implementation of an Optic Flow Algorithm Suitable for Background Oriented Schlieren Technique

A Fast and Accurate Optical Flow Camera for Resource-Constrained Edge Applications

Optical Flow Dataset and Benchmark for Visual Crowd Analysis

NeuFlow: Real-time, High-accuracy Optical Flow Estimation on Robots Using Edge Devices

Hybrid Cost Volume for Memory-Efficient Optical Flow

SciFlow: Empowering Lightweight Optical Flow Models with Self-Cleaning Iterations

An investigation of camera movements and capture techniques on optical flow for real-time rendering and presentation

Robust Optical Flow Computation: A Higher-Order Differential Approach

Memory-Efficient Optical Flow via Radius-Distribution Orthogonal Cost Volume

Optical Flow Estimation Between Images of Different Resolutions Via Variational Method

Neuromorphic Optical Flow and Real-time Implementation with Event Cameras

Real-time Structure Flow

A new algorithm for estimating optic flow for low-level vision systems

FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation

Full Flow: Optical Flow Estimation By Global Optimization over Regular Grids