Beyond the Label Itself: Latent Labels Enhance Semi-supervised Point Cloud Panoptic Segmentation

Yujun Chen,Xin Tan,Zhizhong Zhang,Yanyun Qu,Yuan Xie
DOI: https://doi.org/10.1609/aaai.v38i2.27887
2024-01-01
Abstract:As the exorbitant expense of labeling autopilot datasets and the growingtrend of utilizing unlabeled data, semi-supervised segmentation on point cloudsbecomes increasingly imperative. Intuitively, finding out more “unspokenwords” (i.e., latent instance information) beyond the label itself should behelpful to improve performance. In this paper, we discover two types of latentlabels behind the displayed label embedded in LiDAR and image data. First, inthe LiDAR Branch, we propose a novel augmentation, Cylinder-Mix, which is ableto augment more yet reliable samples for training. Second, in the Image Branch,we propose the Instance Position-scale Learning (IPSL) Module to learn and fusethe information of instance position and scale, which is from a 2D pre-traineddetector and a type of latent label obtained from 3D to 2D projection. Finally,the two latent labels are embedded into the multi-modal panoptic segmentationnetwork. The ablation of the IPSL module demonstrates its robust adaptability,and the experiments evaluated on SemanticKITTI and nuScenes demonstrate thatour model outperforms the state-of-the-art method, LaserMix.
What problem does this paper attempt to address?