Abstract:The accuracy of scene flow is restricted by several challenges such as occlusion and large displacement motion. When occlusion happens, the positions inside the occluded regions lose their corresponding counterparts in preceding and succeeding frames. Large displacement motion will increase the complexity of motion modeling and computation. Moreover, occlusion and large displacement motion are highly related problems in scene flow estimation, e.g., large displacement motion often leads to considerably occluded regions in the scene. An improved dense scene flow method based on red-green-blue-depth (RGB-D) data is proposed in this paper. To handle occlusion, we model the occlusion status for each point in our problem formulation, and jointly estimate the scene flow and occluded regions. To deal with large displacement motion, we employ an over-parameterized scene flow representation to model both the rotation and translation components of the scene flow, since large displacement motion cannot be well approximated using translational motion only. Furthermore, we employ a two-stage optimization procedure for this overparameterized scene flow representation. In the first stage, we propose a new RGB-D PatchMatch method, which is mainly applied in the RGB-D image space to reduce the computational complexity introduced by the large displacement motion. According to the quantitative evaluation based on the Middlebury data set, our method outperforms other published methods. The improved performance is also comprehensively confirmed on the real data acquired by Kinect sensor.

SS-SF: Piecewise 3D Scene Flow Estimation With Semantic Segmentation

Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity.

SF2SE3: Clustering Scene Flow into SE(3)-Motions via Proposal and Selection

Automatic Layered RGB-D Scene Flow Estimation with Optical Flow Field Constraint.

SSF-MOS: Semantic Scene Flow Assisted Moving Object Segmentation for Autonomous Vehicles

Fast Multi-frame Stereo Scene Flow with Motion Segmentation

A Continuous Optimization Approach for Efficient and Accurate Scene Flow

SAFIT: Segmentation-Aware Scene Flow with Improved Transformer

Handling Occlusion and Large Displacement Through Improved RGB-D Scene Flow Estimation

Scene Flow Estimation Based on 3D Local Rigidity Assumption and Depth Map Driven Anisotropic Smoothness

SSRFlow: Semantic-aware Fusion with Spatial Temporal Re-embedding for Real-world Scene Flow

Optical Flow with Semantic Segmentation and Localized Layers

SAMFlow: Eliminating Any Fragmentation in Optical Flow with Segment Anything Model

Multi-Motion Segmentation: Combining Geometric Model-Fitting and Optical Flow for RGB Sensors

UnSAMFlow: Unsupervised Optical Flow Guided by Segment Anything Model

Flow2seg: Motion-Aided Semantic Segmentation

EMR-MSF: Self-Supervised Recurrent Monocular Scene Flow Exploiting Ego-Motion Rigidity

Salient-motion-heuristic Scheme for Fast 3D Optical Flow Estimation Using RGB-D Data

Self-Supervised Scene Flow Estimation with Point-Voxel Fusion and Surface Representation

Semantic Flow: Learning Semantic Field of Dynamic Scenes from Monocular Videos

ScaleFlow++: Robust and Accurate Estimation of 3D Motion from Video