Abstract:Optical flow estimation is crucial for various applications in vision and robotics. As the difficulty of collecting ground truth optical flow in real-world scenarios, most of the existing methods of learning optical flow still adopt synthetic dataset for supervised training or utilize photometric consistency across temporally adjacent video frames to drive the unsupervised learning, where the former typically has issues of generalizability while the latter usually performs worse than the supervised ones. To tackle such challenges, we propose to leverage the geometric connection between optical flow estimation and stereo matching (based on the similarity upon finding pixel correspondences across images) to unify various real-world depth estimation datasets for generating supervised training data upon optical flow. Specifically, we turn the monocular depth datasets into stereo ones via synthesizing virtual disparity, thus leading to the flows along the horizontal direction; moreover, we introduce virtual camera motion into stereo data to produce additional flows along the vertical direction. Furthermore, we propose applying geometric augmentations on one image of an optical flow pair, encouraging the optical flow estimator to learn from more challenging cases. Lastly, as the optical flow maps under different geometric augmentations actually exhibit distinct characteristics, an auxiliary classifier which trains to identify the type of augmentation from the appearance of the flow map is utilized to further enhance the learning of the optical flow estimator. Our proposed method is general and is not tied to any particular flow estimator, where extensive experiments based on various datasets and optical flow estimation models verify its efficacy and superiority.

Unsupervised Learning of Facial Optical Flow via Occlusion-Aware Global-Local Matching.

Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity.

A Dense Optical Flow Registration Algorithm Based on Deep Learning

Learning By Analogy: Reliable Supervision From Transformations For Unsupervised Optical Flow Estimation

Unsupervised Learning of Optical Flow with Patch Consistency and Occlusion Estimation.

NccFlow: Unsupervised Learning of Optical Flow With Non-occlusion from Geometry

Unsupervised Learning Optical Flow in Multi-frame Dynamic Environment Using Temporal Dynamic Modeling

Skin the sheep not only once: Reusing Various Depth Datasets to Drive the Learning of Optical Flow

Self-Attention-Based Multiscale Feature Learning Optical Flow with Occlusion Feature Map Prediction

An Unsupervised Optical Flow Estimation For LiDAR Image Sequences

Global Matching with Overlapping Attention for Optical Flow Estimation

Rethinking Optical Flow from Geometric Matching Consistent Perspective

Unsupervised Learning of Depth, Optical Flow and Pose With Occlusion From 3D Geometry

Unsupervised Deep Learning for Optical Flow Estimation.

SKFlow: Learning Optical Flow with Super Kernels

Optical Flow boosts Unsupervised Localization and Segmentation

Survey on Unsupervised Learning Methods for Optical Flow Estimation

Optical Flow as Spatial-Temporal Attention Learners

Robust Non-Local TV- $L^{1}$ Optical Flow Estimation with Occlusion Detection

Self-Supervised Monocular Scene Flow Estimation

Joint facial action unit recognition and self-supervised optical flow estimation