Abstract:Scene flow estimation is the task to predict the point-wise or pixel-wise 3-D displacement vector between two consecutive frames of point clouds or images, which has important application in fields such as service robots and autonomous driving. Although many previous works have explored greatly on scene flow estimation based on point clouds, there are two problems that have not been noticed or well solved before: 1) points of adjacent frames in repetitive patterns tend to be wrongly associated due to similar spatial structure in their neighborhoods and 2) scene flow between adjacent frames of point clouds with long-distance movement tends to be inaccurately estimated. To solve the first problem, a novel context-aware set convolution layer is proposed in this article to exploit contextual structure information of Euclidean space and learn soft aggregation weights for local point features. This design is inspired by human perception of contextual structure information during scene understanding with repetitive patterns. For the second problem, an explicit residual flow learning structure is proposed in the residual flow refinement layer to cope with long-distance movement. The experiments and ablation study on FlyingThings3D and Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) scene flow datasets demonstrate the effectiveness of each proposed component. The qualitative results show that the problems of ambiguous interframe association and long-distance movement estimation are well handled. Quantitative results on both FlyingThings3D and KITTI scene flow datasets show that the proposed method achieves state-of-the-art performance, surpassing all previous works to the best of our knowledge by at least 26.22% on FlyingThings3D dataset and 25.42% on KITTI scene flow dataset for 3-D endpoint error (EPE3D) metric. The source codes are available at: https://github.com/IRMVLab/CARFlow.

Regularizing Self-supervised 3D Scene Flows with Surface Awareness and Cyclic Consistency

Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity.

Weakly Supervised Learning of Rigid 3D Scene Flow

Let-It-Flow: Simultaneous Optimization of 3D Flow and Object Clustering

RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior

Self-Supervised 3D Scene Flow Estimation and Motion Prediction using Local Rigidity Prior

Unsupervised Learning of 3D Scene Flow with 3D Odometry Assistance

Residual 3D Scene Flow Learning with Context-Aware Feature Extraction

Self-Supervised Learning of Non-Rigid Residual Flow and Ego-Motion

Adversarial Self-Supervised Scene Flow Estimation

Unsupervised Learning of 3D Scene Flow from Monocular Camera

Self-Supervised Flow Estimation using Geometric Regularization with Applications to Camera Image and Grid Map Sequences

Self-Supervised Scene Flow Estimation with Point-Voxel Fusion and Surface Representation

Self-supervised Learning of Occlusion Aware Flow Guided 3D Geometry Perception with Adaptive Cross Weighted Loss from Monocular Videos

SeFlow: A Self-Supervised Scene Flow Method in Autonomous Driving

EMR-MSF: Self-Supervised Recurrent Monocular Scene Flow Exploiting Ego-Motion Rigidity

RAFT-MSF: Self-Supervised Monocular Scene Flow Using Recurrent Optimizer

SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self

Do not trust the neighbors! Adversarial Metric Learning for Self-Supervised Scene Flow Estimation

Self-Supervised Monocular Scene Flow Estimation