Abstract:Optical flow represents vector motion of targets, and its estimation is a hot research in computer vision. The traditional prior-based methods build mathematical models using different regularizers on object characteristics. These methods are straightforward and easily understood, but they suffer from a high computational burden. Deep learning based approaches greatly improve their efficiency, but their filter sizes are fixed and receptive fields are large. To solve these problems, this paper proposes a novel recurrent feature refinement network based on short-term dense connection modules for optical flow estimation (RFRFlow), including the following key parts: firstly, we utilize short-term dense concatenate modules (STDC) as a backbone part for constructing a feature extracting network which can preserve rich details and use global averaging pooling to extract global context information with a large receptive field. Secondly, a 4D correlation volume is constructed using context features of image pairs to extract multi-scale 4D correlation information. Thirdly, a motion feature encoder is reported to transfer the multi-scale 4D correlation information into 2D motion features. Finally, 2D context features and 2D motion features are input into a global motion aggregation (GMA) decoder for pursuing the final optical flow. In the experiments, we firstly present and analyze the results of ablation studies on the selection of batch size and image size. Then, both quantitative and qualitative results on Sintel and KITTI datasets are compared, showing that the proposed RFRFlow is effective and even performs better than the state-of-the-art optical flow estimation methods on performance improvement and objects’ shape preservation.

Hierarchical Coherency Sensitive Hashing and Interpolation with RANSAC for Large Displacement Optical Flow

Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity.

HMAFlow: Learning More Accurate Optical Flow via Hierarchical Motion Field Alignment

Flow Fields: Dense Correspondence Fields for Highly Accurate Large Displacement Optical Flow Estimation

RFRFlow: Recurrent feature refinement network for optical flow estimation

HCRF-Flow: Scene Flow from Point Clouds with Continuous High-order CRFs and Position-aware Flow Embedding

STDC‐Flow: Large Displacement Flow Field Estimation Using Similarity Transformation‐based Dense Correspondence

Neighbor Correspondence Matching for Flow-based Video Frame Synthesis.

CCMR: High Resolution Optical Flow Estimation via Coarse-to-Fine Context-Guided Motion Reasoning

Self-Attention-Based Multiscale Feature Learning Optical Flow with Occlusion Feature Map Prediction

Optical Flow Super-Resolution Based on Image Guidence Using Convolutional Neural Network

Exploring Neighbor Correspondence Matching for Multiple-Hypotheses Video Frame Synthesis

OCAI: Improving Optical Flow Estimation by Occlusion and Consistency Aware Interpolation

HMFlow: Hybrid Matching Optical Flow Network for Small and Fast-Moving Objects

FPCR-Net: Feature pyramidal correlation and residual reconstruction for optical flow estimation

Optical Flow Estimation Via Motion Feature Recovery

ResFlow: Fine-tuning Residual Optical Flow for Event-based High Temporal Resolution Motion Estimation

Motion Estimation for Large Displacements and Deformations

Optical Flow Estimation from Layered Nearest Neighbor Flow Fields.

FlowNAS: Neural Architecture Search for Optical Flow Estimation

CSFlow: Learning Optical Flow via Cross Strip Correlation for Autonomous Driving