Abstract:For visual estimation of optical flow, a crucial function for many vision tasks, unsupervised learning, using the supervision of view synthesis has emerged as a promising alternative to supervised methods, since ground-truth flow is not readily available in many cases. However, unsupervised learning is likely to be unstable when pixel tracking is lost due to occlusion and motion blur, or the pixel matching is impaired due to variation in image content and spatial structure over time. In natural environments, dynamic occlusion or object variation is a relatively slow temporal process spanning several frames. We, therefore, explore the optical flow estimation from multiple-frame sequences of dynamic scenes, whereas most of the existing unsupervised approaches are based on temporal static models. We handle the unsupervised optical flow estimation with a temporal dynamic model by introducing a spatial-temporal dual recurrent block based on the predictive coding structure, which feeds the previous high-level motion prior to the current optical flow estimator. Assuming temporal smoothness of optical flow, we use motion priors of the adjacent frames to provide more reliable supervision of the occluded regions. To grasp the essence of challenging scenes, we simulate various scenarios across long sequences, including dynamic occlusion, content variation, and spatial variation, and adopt self-supervised distillation to make the model understand the object's motion patterns in a prolonged dynamic environment. Experiments on KITTI 2012, KITTI 2015, Sintel Clean, and Sintel Final datasets demonstrate the effectiveness of our methods on unsupervised optical flow estimation. The proposal achieves state-of-the-art performance with advantages in memory overhead.

Unsupervised Learning of Optical Flow with Patch Consistency and Occlusion Estimation.

Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity.

A Dense Optical Flow Registration Algorithm Based on Deep Learning

NccFlow: Unsupervised Learning of Optical Flow With Non-occlusion from Geometry

Learning By Analogy: Reliable Supervision From Transformations For Unsupervised Optical Flow Estimation

Unsupervised Deep Learning for Optical Flow Estimation.

MRDFlow: Unsupervised Optical Flow Estimation Network With Multi-Scale Recurrent Decoder

STFlow: Self-Taught Optical Flow Estimation Using Pseudo Labels

Self-Attention-Based Multiscale Feature Learning Optical Flow with Occlusion Feature Map Prediction

Unsupervised Optical Flow Estimation Based on Improved Feature Pyramid

Masked Label Learning For Optical Flow Regression

Unsupervised Learning Optical Flow in Multi-frame Dynamic Environment Using Temporal Dynamic Modeling

Unsupervised Learning for Optical Flow Estimation Using Pyramid Convolution LSTM

An Unsupervised Optical Flow Estimation For LiDAR Image Sequences

Survey on Unsupervised Learning Methods for Optical Flow Estimation

Optical Flow Estimation Using a Non-Local Convolutional Network.

Self-Supervised Monocular Scene Flow Estimation

Recurrent Partial Kernel Network for Efficient Optical Flow Estimation

FlowDepth: Decoupling Optical Flow for Self-Supervised Monocular Depth Estimation

ReFlowNet: Revisiting Coarse-to-fine Learning of Optical Flow

Breaking of brightness consistency in optical flow with a lightweight CNN network