Abstract:Occlusions between consecutive frames have long posed a significant challenge in optical flow estimation. The inherent ambiguity introduced by occlusions directly violates the brightness constancy constraint and considerably hinders pixel-to-pixel matching. To address this issue, multi-frame optical flow methods leverage adjacent frames to mitigate the local ambiguity. Nevertheless, prior multi-frame methods predominantly adopt recursive flow estimation, resulting in a considerable computational overlap. In contrast, we propose a streamlined in-batch framework that eliminates the need for extensive redundant recursive computations while concurrently developing effective spatio-temporal modeling approaches under in-batch estimation constraints. Specifically, we present a Streamlined In-batch Multi-frame (SIM) pipeline tailored to video input, attaining a similar level of time efficiency to two-frame networks. Furthermore, we introduce an efficient Integrative Spatio-temporal Coherence (ISC) modeling method for effective spatio-temporal modeling during the encoding phase, which introduces no additional parameter overhead. Additionally, we devise a Global Temporal Regressor (GTR) that effectively explores temporal relations during decoding. Benefiting from the efficient SIM pipeline and effective modules, StreamFlow not only excels in terms of performance on the challenging KITTI and Sintel datasets, with particular improvement in occluded areas but also attains a remarkable 63.82% enhancement in speed compared with previous multi-frame methods. The code will be available soon at https://github.com/littlespray/StreamFlow.

LLA-FLOW: A Lightweight Local Aggregation on Cost Volume for Optical Flow Estimation

Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity.

A Dense Optical Flow Registration Algorithm Based on Deep Learning

Local texture based optical flow for complex brightness variations

Adaptive Fractional-Order Multi-Scale Optimization TV-L1 Optical Flow Algorithm

Learning By Analogy: Reliable Supervision From Transformations For Unsupervised Optical Flow Estimation

Robust Non-Local TV- $L^{1}$ Optical Flow Estimation with Occlusion Detection

A Robust Optical Flow Computation

LCIF-Net: Local criss-cross attention based optical flow method using multi-scale image features and feature pyramid

Optical Flow Estimation from Layered Nearest Neighbor Flow Fields.

An Unsupervised Optical Flow Estimation For LiDAR Image Sequences

HMAFlow: Learning More Accurate Optical Flow via Hierarchical Motion Field Alignment

Hybrid Cost Volume for Memory-Efficient Optical Flow

SciFlow: Empowering Lightweight Optical Flow Models with Self-Cleaning Iterations

Optical Flow Estimation Via Motion Feature Recovery

Memory-Efficient Optical Flow via Radius-Distribution Orthogonal Cost Volume

Self-Attention-Based Multiscale Feature Learning Optical Flow with Occlusion Feature Map Prediction

StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences

STDC‐Flow: Large Displacement Flow Field Estimation Using Similarity Transformation‐based Dense Correspondence

Learning Optical Flow from a Few Matches

MemFlow: Optical Flow Estimation and Prediction with Memory