Abstract:Optical flow estimation is crucial for various applications in vision and robotics. As the difficulty of collecting ground truth optical flow in real-world scenarios, most of the existing methods of learning optical flow still adopt synthetic dataset for supervised training or utilize photometric consistency across temporally adjacent video frames to drive the unsupervised learning, where the former typically has issues of generalizability while the latter usually performs worse than the supervised ones. To tackle such challenges, we propose to leverage the geometric connection between optical flow estimation and stereo matching (based on the similarity upon finding pixel correspondences across images) to unify various real-world depth estimation datasets for generating supervised training data upon optical flow. Specifically, we turn the monocular depth datasets into stereo ones via synthesizing virtual disparity, thus leading to the flows along the horizontal direction; moreover, we introduce virtual camera motion into stereo data to produce additional flows along the vertical direction. Furthermore, we propose applying geometric augmentations on one image of an optical flow pair, encouraging the optical flow estimator to learn from more challenging cases. Lastly, as the optical flow maps under different geometric augmentations actually exhibit distinct characteristics, an auxiliary classifier which trains to identify the type of augmentation from the appearance of the flow map is utilized to further enhance the learning of the optical flow estimator. Our proposed method is general and is not tied to any particular flow estimator, where extensive experiments based on various datasets and optical flow estimation models verify its efficacy and superiority.

Determining Relative Geometry Of Cameras From Normal Flows

Stereo Camera Motion Estimation Based on Euclidean Reconstruction

Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity.

Motion and Structure from Event-based Normal Flow

3D Facial Geometry Recovery Via Group-Wise Optical Flow

Relative Pose Estimation for Multi-Camera Systems from Point Correspondences with Scale Ratio

Monitoring activities from multiple video streams: establishing a common coordinate frame

Stereoscopic video conversion based on depth tracking

OptFlowCam: A 3D‐Image‐Flow‐Based Metric in Camera Space for Camera Paths in Scenes with Extreme Scale Variations

1D Camera Geometry and Its Application to Circular Motion Estimation.

Determining dense velocity fields for fluid images based on affine motion

Real-time Halfway Domain Reconstruction of Motion and Geometry

Cameras Viewing Cameras Geometry

Self-supervised Learning of Monocular 3D Geometry Understanding with Two- and Three-View Geometric Constraints

ScaleFlow++: Robust and Accurate Estimation of 3D Motion from Video

3D Reconstruction from public webcams

Relative Pose for Nonrigid Multi-Perspective Cameras: The Static Case

A New Non-Linear Algorithm for 3D Reconstruction from Straight-line Optical Flow

Skin the sheep not only once: Reusing Various Depth Datasets to Drive the Learning of Optical Flow

Relative Pose Estimation for Stereo Rolling Shutter Cameras