EgoFlowNet: Non-Rigid Scene Flow from Point Clouds with Ego-Motion Support

Ramy Battrawy,René Schuster,Didier Stricker

2024-07-03

Abstract:Recent weakly-supervised methods for scene flow estimation from LiDAR point clouds are limited to explicit reasoning on object-level. These methods perform multiple iterative optimizations for each rigid object, which makes them vulnerable to clustering robustness. In this paper, we propose our EgoFlowNet - a point-level scene flow estimation network trained in a weakly-supervised manner and without object-based abstraction. Our approach predicts a binary segmentation mask that implicitly drives two parallel branches for ego-motion and scene flow. Unlike previous methods, we provide both branches with all input points and carefully integrate the binary mask into the feature extraction and losses. We also use a shared cost volume with local refinement that is updated at multiple scales without explicit clustering or rigidity assumptions. On realistic KITTI scenes, we show that our EgoFlowNet performs better than state-of-the-art methods in the presence of ground surface points.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to estimate the scene flow in LiDAR point clouds, especially for non - rigid scenes, and support ego - motion. Specifically, the existing weakly - supervised methods are usually limited to explicit reasoning at the object level when dealing with scene flow estimation in LiDAR point clouds. These methods perform multiple iterative optimizations on each rigid object, which makes them vulnerable in terms of clustering robustness. In addition, these methods perform poorly when dealing with ground points, occlusions, and outliers.

To overcome these problems, the authors propose EgoFlowNet, a point - based scene flow estimation network that adopts weakly - supervised training and does not rely on object abstraction. The main contributions of EgoFlowNet are as follows:

1. **Multi - task neural network architecture**: EgoFlowNet can directly jointly estimate binary segmentation masks, ego - motion, and scene flow from the original point clouds.
2. **Hybrid feature extraction**: It combines hybrid feature extraction and hybrid deformation layers, and integrates the binary mask into the feature extraction and loss function to obtain robust scene flow estimation.
3. **Point - level refinement**: It refines the scene flow at the point level without relying on an explicit clustering mechanism or the rigid assumption of dynamic objects.
4. **Performance in difficult real - world LiDAR scenes**: In difficult real - world LiDAR scenes containing ground points, occlusions, and outliers, EgoFlowNet outperforms the recent clustering - based methods.

Through these improvements, EgoFlowNet can provide more accurate and robust scene flow estimation when dealing with non - rigid scenes and complex environments.