Abstract:Weakly supervised point cloud semantic segmentation methods that require 1% or fewer labels with the aim of realizing almost the same performance as fully supervised approaches have recently attracted extensive research attention. A typical solution in this framework is to use self-training or pseudo-labeling to mine the supervision from the point cloud itself while ignoring the critical information from images. In fact, cameras widely exist in LiDAR scenarios, and this complementary information seems to be highly important for 3D applications. In this paper, we propose a novel cross-modality weakly supervised method for 3D segmentation that incorporates complementary information from unlabeled images. We design a dual-branch network equipped with an active labeling strategy to maximize the power of tiny parts of labels and to directly realize 2D-to-3D knowledge transfer. Afterward, we establish a cross-modal self-training framework, which iterates between parameter updating and pseudolabel estimation. In the training phase, we propose cross-modal association learning to mine complementary supervision from images by reinforcing the cycle consistency between 3D points and 2D superpixels. In the pseudolabel estimation phase, a pseudolabel self-rectification mechanism is derived to filter noisy labels, thus providing more accurate labels for the networks to be fully trained. The extensive experimental results demonstrate that our method even outperforms the state-of-the-art fully supervised competitors with less than 1% actively selected annotations.

Weakly Supervised 3D Semantic Segmentation Using Cross-Image Consensus and Inter-Voxel Affinity Relations

3D Multiple-Contextual ROI-Attention Network for Efficient and Accurate Volumetric Medical Image Segmentation.

3D Weakly Supervised Semantic Segmentation with 2D Vision-Language Guidance

Weakly Supervised 3D Segmentation Via Receptive-Driven Pseudo Label Consistency and Structural Consistency.

3D Guided Weakly Supervised Semantic Segmentation

Weakly Supervised 3D Instance Segmentation without Instance-level Annotations

CIAN: Cross-Image Affinity Net for Weakly Supervised Semantic Segmentation

Dense Supervision Propagation for Weakly Supervised Semantic Segmentation on 3D Point Clouds

Supervoxel Convolution for Online 3D Semantic Segmentation

Weakly Supervised Classification Model for Zero‐shot Semantic Segmentation

3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

Weakly Supervised Semantic Segmentation Based on Co-segmentation.

Weakly-supervised Semantic Segmentation with Superpixel Guided Local and Global Consistency

Weakly Supervised Semantic Segmentation Based on Web Image Co-segmentation

Learning 3D Semantic Segmentation with only 2D Image Supervision

Sparse Reconstruction for Weakly Supervised Semantic Segmentation.

Weakly Supervised Semantic Segmentation of Point Cloud Scenes Using Boundary-based Feature Aggregation

Deep learning based supervised semantic segmentation of Electron Cryo-Subtomograms

Weakly Supervised Semantic Segmentation for Social Images

Weakly-Supervised Point Cloud Semantic Segmentation Based on Dilated Region

Image Understands Point Cloud: Weakly Supervised 3D Semantic Segmentation via Association Learning