Abstract:Weakly supervised point cloud semantic segmentation methods that require 1% or fewer labels with the aim of realizing almost the same performance as fully supervised approaches have recently attracted extensive research attention. A typical solution in this framework is to use self-training or pseudo-labeling to mine the supervision from the point cloud itself while ignoring the critical information from images. In fact, cameras widely exist in LiDAR scenarios, and this complementary information seems to be highly important for 3D applications. In this paper, we propose a novel cross-modality weakly supervised method for 3D segmentation that incorporates complementary information from unlabeled images. We design a dual-branch network equipped with an active labeling strategy to maximize the power of tiny parts of labels and to directly realize 2D-to-3D knowledge transfer. Afterward, we establish a cross-modal self-training framework, which iterates between parameter updating and pseudolabel estimation. In the training phase, we propose cross-modal association learning to mine complementary supervision from images by reinforcing the cycle consistency between 3D points and 2D superpixels. In the pseudolabel estimation phase, a pseudolabel self-rectification mechanism is derived to filter noisy labels, thus providing more accurate labels for the networks to be fully trained. The extensive experimental results demonstrate that our method even outperforms the state-of-the-art fully supervised competitors with less than 1% actively selected annotations.

Image-driven Unsupervised 3D Model Co-Segmentation

3D Graph-S<SUP>2</SUP>Net: Shape-Aware Self-ensembling Network for Semi-supervised Segmentation with Bilateral Graph Convolution

Unsupervised Salient Object Segmentation From Color Images

Learning 3D Semantic Segmentation with only 2D Image Supervision

Scribble-Based 3D Shape Segmentation via Weakly-Supervised Learning

Superpoint-guided Semi-supervised Semantic Segmentation of 3D Point Clouds

Semi-Supervised Learning for Visual Bird's Eye View Semantic Segmentation

Unsupervised 3D Shape Segmentation and Co-Segmentation Via Deep Learning.

3D Guided Weakly Supervised Semantic Segmentation

Unsupervised Universal Image Segmentation

UnScene3D: Unsupervised 3D Instance Segmentation for Indoor Scenes

Semi-supervised 3D shape segmentation with multilevel consistency and part substitution

Segment3D: Learning Fine-Grained Class-Agnostic 3D Segmentation without Manual Labels

Leveraging Large-Scale Pretrained Vision Foundation Models for Label-Efficient 3D Point Cloud Segmentation

Self-supervised learning via inter-modal reconstruction and feature projection networks for label-efficient 3D-to-2D segmentation

Image Understands Point Cloud: Weakly Supervised 3D Semantic Segmentation via Association Learning

Unsupervised Multi-Class Co-Segmentation via Joint-Cut Over $L_{1}$ -Manifold Hyper-Graph of Discriminative Image Regions

Semi-Supervised 3D Shape Segmentation via Self Refining

Unsupervised Part Segmentation through Disentangling Appearance and Shape

Deep Learning based 3D Segmentation: A Survey

Semantic 3D Reconstruction with Learning MVS and 2D Segmentation of Aerial Images