Abstract:Optical flow represents vector motion of targets, and its estimation is a hot research in computer vision. The traditional prior-based methods build mathematical models using different regularizers on object characteristics. These methods are straightforward and easily understood, but they suffer from a high computational burden. Deep learning based approaches greatly improve their efficiency, but their filter sizes are fixed and receptive fields are large. To solve these problems, this paper proposes a novel recurrent feature refinement network based on short-term dense connection modules for optical flow estimation (RFRFlow), including the following key parts: firstly, we utilize short-term dense concatenate modules (STDC) as a backbone part for constructing a feature extracting network which can preserve rich details and use global averaging pooling to extract global context information with a large receptive field. Secondly, a 4D correlation volume is constructed using context features of image pairs to extract multi-scale 4D correlation information. Thirdly, a motion feature encoder is reported to transfer the multi-scale 4D correlation information into 2D motion features. Finally, 2D context features and 2D motion features are input into a global motion aggregation (GMA) decoder for pursuing the final optical flow. In the experiments, we firstly present and analyze the results of ablation studies on the selection of batch size and image size. Then, both quantitative and qualitative results on Sintel and KITTI datasets are compared, showing that the proposed RFRFlow is effective and even performs better than the state-of-the-art optical flow estimation methods on performance improvement and objects’ shape preservation.

GTEA: Guided Taylor Expansion Approximation Network for Optical Flow Estimation

Recurrent Graph Optimal Transport for Learning 3D Flow Motion in Particle Tracking

DeepPTV: Particle Tracking Velocimetry for Complex Flow Motion via Deep Neural Networks

Particle image velocimetry based on a deep neural network

RFRFlow: Recurrent feature refinement network for optical flow estimation

FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation

Video Enhancement with Task-Oriented Flow

Deep Networks for Image Motion Estimation

Optical Flow Estimation Using a Non-Local Convolutional Network.

Global Matching with Overlapping Attention for Optical Flow Estimation

Rethinking Optical Flow from Geometric Matching Consistent Perspective

FlowNAS: Neural Architecture Search for Optical Flow Estimation

Context-Aware Iteration Policy Network for Efficient Optical Flow Estimation

Exploiting Optical Flow Guidance for Transformer-Based Video Inpainting

OFR-Net: Optical Flow Refinement with a Pyramid Dense Residual Network.

TransFlow: Transformer as Flow Learner

Deep Networks for Image-to-Image Prediction

EGOF-Net: Epipolar Guided Optical Flow Network for Unrectified Stereo Matching

GMFlow: Learning Optical Flow Via Global Matching

Recurrent Partial Kernel Network for Efficient Optical Flow Estimation

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks