Abstract:Semisupervised semantic segmentation aims to effectively leverage both unlabeled and scare labeled images, reducing the reliance on labor-intensive pixel-level labeling for extensive training processes. The leading semisupervised learning method, consistency regularization, employs weak and strong data augmentations to diversify input representations. Ultimately the model is compelled to maintain consistent predictions across different input views, thus boosting the model's generalization. However, previous methods suffered from limited input representation space introduced by linear transformations such as cutmix. To address such issue, a consistency regularization based on masked image modeling (MIM) called MIMSeg is proposed to achieve accurate segmentation with limited labeled images. First, MIM pixel-wise perception with ViT encoder-decoder lays the foundation for expanding the data representation space. Second, collaborating with weak data augmentations, two MIM-related strong data augmentations are developed to generate more challenging input views for consistent predictions. Precisely, weak data augmentations are employed to replicate input views from various perspectives while a controllable generative strong data augmentation called masked image reconstruction (MIR) is crafted to simulate multiple imaging diversity while preserving the original semantic information intact. In addition, a more severe strong data augmentation masked context perturbation (MCP) is designed to further generate more challenging input views and alleviate semantic deficiency via masked category prediction. Leveraging the MIM perception and two MIM-related strong data augmentations, the model is compelled to achieve consistency predictions across diverse input views from weak data augmentations, MIR and MCP. These components result in the generation of more stable pixel-level pseudo-labels and facilitate collaborative training between unlabeled and labeled images. Extensive experiments have shown that MIMSeg can achieve state-of-the-art performance in pixel-level prediction with very limited sample annotations.

Learning Multiscale Consistency for Self-supervised Electron Microscopy Instance Segmentation

Semi-Supervised Neuron Segmentation via Reinforced Consistency Learning

Addressing Distribution Mismatch for Effective Semi-Supervised Neuron Segmentation

Self-supervised Learning of Morphological Representation for 3D EM Segments with Cluster-Instance Correlations.

Weakly-Supervised Cross-Domain Segmentation of Electron Microscopy with Sparse Point Annotation

Domain Adaptive Mitochondria Segmentation Via Enforcing Inter-Section Consistency

Dual-consistency guidance semi-supervised medical image segmentation with low-level detail feature augmentation

Hierarchical Level Features Based Trainable Segmentation for Electron Microscopy Images

Curriculum Consistency Learning and Multi-Scale Contrastive Constraint in Semi-Supervised Medical Image Segmentation

Semi-Supervised Segmentation of Mitochondria from Electron Microscopy Images Using Spatial Continuity

Consistency learning with dynamic weighting and class-agnostic regularization for semi-supervised medical image segmentation

Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing

Consistency Regularization Based on Masked Image Modeling for Semi-Supervised Remote Sensing Semantic Segmentation

SDMI-Net

Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning

Semi-supervised segmentation of cardiac chambers from LGE-CMR using feature consistency awareness

Multi-Consistency Training for Semi-Supervised Medical Image Segmentation

Beyond Low-Dimensional Features: Enhancing Semi-Supervised Medical Image Semantic Segmentation with Advanced Consistency Learning Techniques

Sequential semi-supervised segmentation for serial electron microscopy image with small number of labels

Exploring Inherent Consistency for Semi-supervised Anatomical Structure Segmentation in Medical Imaging

Triple-task mutual consistency for semi-supervised 3D medical image segmentation