Abstract:Label-efficient segmentation aims to perform effective segmentation on input data using only sparse and limited ground-truth labels for training. This topic is widely studied in 3D point cloud segmentation due to the difficulty of annotating point clouds densely, while it is also essential for cost-effective segmentation on 2D images. Until recently, pseudo-labels have been widely employed to facilitate training with limited ground-truth labels, and promising progress has been witnessed in both the 2D and 3D segmentation. However, existing pseudo-labeling approaches could suffer heavily from the noises and variations in unlabelled data, which would result in significant discrepancies between generated pseudo-labels and current model predictions during training. We analyze that this can further confuse and affect the model learning process, which shows to be a shared problem in label-efficient learning across both 2D and 3D modalities. To address this issue, we propose a novel learning strategy to regularize the pseudo-labels generated for training, thus effectively narrowing the gaps between pseudo-labels and model predictions. More specifically, our method introduces an Entropy Regularization loss and a Distribution Alignment loss for label-efficient learning, resulting in an ERDA learning strategy. Interestingly, by using KL distance to formulate the distribution alignment loss, ERDA reduces to a deceptively simple cross-entropy-based loss which optimizes both the pseudo-label generation module and the segmentation model simultaneously. In addition, we innovate in the pseudo-label generation to make our ERDA consistently effective across both 2D and 3D data modalities for segmentation. Enjoying simplicity and more modality-agnostic pseudo-label generation, our method has shown outstanding performance in fully utilizing all unlabeled data points for training across ...

Unsupervised learning of finite mixtures using entropy regularization and its application to image segmentation

Entropy Regularization, Automatic Model Selection, And Unsupervised Image Segmentation

Asymptotic Convergence Properties of Entropy Regularized Likelihood Learning on Finite Mixtures with Automatic Model Selection

Unsupervised image segmentation using an iterative entropy regularized likelihood learning algorithm

A Semi-Supervised Learning Algorithm on Gaussian Mixture with Automatic Model Selection

Generalized Competitive Learning of Gaussian Mixture Models

Unsupervised Image Categorization Using Constrained Entropy-Regularized Likelihood Learning with Pairwise Constraints

Letters: An iterative algorithm for entropy regularized likelihood learning on Gaussian mixture with automatic model selection

Entropy Regularized Likelihood Learning on Gaussian Mixture: Two Gradient Implementations for Automatic Model Selection

Unsupervised Learning of Gaussian Mixture Model with Application to Image Segmentation

On Semi-Supervised Learning Genetic-Based And Deterministic Annealing Em Algorithm For Dirichlet Mixture Models

A generalized competitive learning algorithm on gaussian mixture with automatic model selection

An Iterative Entropy Regularized Likelihood Learning Algorithm for Cluster Analysis with the Number of Clusters Automatically Detected

EDRL: Entropy-guided disentangled representation learning for unsupervised domain adaptation in semantic segmentation

A gradient entropy regularized likelihood learning algorithm on gaussian mixture with automatic model selection

Towards Modality-agnostic Label-efficient Segmentation with Entropy-Regularized Distribution Alignment

All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation

Renyi entropy penalized learning algorithm for Gaussian mixture with automated model selection

An Evolutional Learning Algorithm Based on Weighted Likelihood for Image Segmentation

ESL: Entropy-guided Self-supervised Learning for Domain Adaptation in Semantic Segmentation

Neutral Cross-Entropy Loss Based Unsupervised Domain Adaptation for Semantic Segmentation