Abstract:For visual estimation of optical flow, a crucial function for many vision tasks, unsupervised learning, using the supervision of view synthesis has emerged as a promising alternative to supervised methods, since ground-truth flow is not readily available in many cases. However, unsupervised learning is likely to be unstable when pixel tracking is lost due to occlusion and motion blur, or the pixel matching is impaired due to variation in image content and spatial structure over time. In natural environments, dynamic occlusion or object variation is a relatively slow temporal process spanning several frames. We, therefore, explore the optical flow estimation from multiple-frame sequences of dynamic scenes, whereas most of the existing unsupervised approaches are based on temporal static models. We handle the unsupervised optical flow estimation with a temporal dynamic model by introducing a spatial-temporal dual recurrent block based on the predictive coding structure, which feeds the previous high-level motion prior to the current optical flow estimator. Assuming temporal smoothness of optical flow, we use motion priors of the adjacent frames to provide more reliable supervision of the occluded regions. To grasp the essence of challenging scenes, we simulate various scenarios across long sequences, including dynamic occlusion, content variation, and spatial variation, and adopt self-supervised distillation to make the model understand the object's motion patterns in a prolonged dynamic environment. Experiments on KITTI 2012, KITTI 2015, Sintel Clean, and Sintel Final datasets demonstrate the effectiveness of our methods on unsupervised optical flow estimation. The proposal achieves state-of-the-art performance with advantages in memory overhead.

MESD: Exploring Optical Flow Assessment on Edge of Motion Objects with Motion Edge Structure Difference

Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity.

Optical Flow Estimation Via Motion Feature Recovery

An effective motion object detection method using optical flow estimation under a moving camera

HMAFlow: Learning More Accurate Optical Flow via Hierarchical Motion Field Alignment

Direct Method for Motion Estimation and Structure Reconstruction Based on Optical Flow

Joint Self-supervised Depth and Optical Flow Estimation towards Dynamic Objects

A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them

Rethinking Optical Flow from Geometric Matching Consistent Perspective

Robust Optical Flow Computation: A Higher-Order Differential Approach

Self-Attention-Based Multiscale Feature Learning Optical Flow with Occlusion Feature Map Prediction

Guided Filtering: Toward Edge-Preserving for Optical Flow

A PDE-Based Method for Optical Flow Estimation

Research on Motion Estimation Algorithm in Image Segmentation

A Robust Optical Flow Computation

A Dense Optical Flow-Based Feature Matching Approach in Visual Odometry

Joint direct estimation of 3D geometry and 3D motion using spatio temporal gradients

An Unsupervised Optical Flow Estimation For LiDAR Image Sequences

ScaleFlow++: Robust and Accurate Estimation of 3D Motion from Video

FlowDepth: Decoupling Optical Flow for Self-Supervised Monocular Depth Estimation

Unsupervised Learning Optical Flow in Multi-frame Dynamic Environment Using Temporal Dynamic Modeling