Abstract:Semantic segmentation has achieved huge progress via adopting deep Fully Convolutional Networks (FCN). However, the performance of FCN-based models severely rely on the amounts of pixel-level annotations which are expensive and time-consuming. Considering that bounding boxes also contain abundant semantic and objective information, an intuitive solution is to learn the segmentation with weak supervisions from the bounding boxes. How to make full use of the class-level and region-level supervisions from bounding boxes to estimate the uncertain regions is the critical challenge for the weakly supervised learning task. In this paper, we propose a mixture model to address this problem. First, we introduce a box-driven class-wise masking model (BCM) to remove irrelevant regions of each class. Moreover, based on the pixel-level segment proposal generated from the bounding box supervision, we calculate the mean filling rates of each class to serve as an important prior cue to guide the model ignoring the wrongly labeled pixels in proposals. To realize the more fine-grained supervision at instance-level, we further propose the anchor-based filling rate shifting module. Unlike previous methods that directly train models with the generated noisy proposals, our method can adjust the model learning dynamically with the adaptive segmentation loss. Thus it can help reduce the negative impacts from wrongly labeled proposals. Besides, based on the learned high-quality proposals with above pipeline, we explore to further boost the performance through two-stage learning. The proposed method is evaluated on the challenging PASCAL VOC 2012 benchmark and achieves 74.9 % and 76.4 % mean IoU accuracy under weakly and semi-supervised modes, respectively. Extensive experimental results show that the proposed method is effective and is on par with, or even better than current state-of-the-art methods.

MaskMentor: Unlocking the Potential of Masked Self-Teaching for Missing Modality RGB-D Semantic Segmentation

When Masked Image Modeling Meets Source-free Unsupervised Domain Adaptation: Dual-Level Masked Network for Semantic Segmentation

Mask-guided Modality Difference Reduction Network for RGB-T Semantic Segmentation

Masked Image Modeling Boosting Semi-Supervised Semantic Segmentation

Mx2M: Masked Cross-Modality Modeling in Domain Adaptation for 3D Semantic Segmentation

Mitigating Modality Discrepancies for RGB-T Semantic Segmentation

Masked Supervised Learning for Semantic Segmentation

Rethinking mask heads for partially supervised instance segmentation

XMask3D: Cross-modal Mask Reasoning for Open Vocabulary 3D Semantic Segmentation

Masked Momentum Contrastive Learning for Zero-shot Semantic Understanding

SegMind: Semisupervised Remote Sensing Image Semantic Segmentation With Masked Image Modeling and Contrastive Learning Method

PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detection

Missing Modality Robustness in Semi-Supervised Multi-Modal Semantic Segmentation

SAM-Guided Masked Token Prediction for 3D Scene Understanding

CM-MaskSD: Cross-Modality Masked Self-Distillation for Referring Image Segmentation

MaskBEV: Towards A Unified Framework for BEV Detection and Map Segmentation

Mask-Adapter: The Devil is in the Masks for Open-Vocabulary Segmentation

Weakly Supervised Semantic Segmentation via Box-Driven Masking and Filling Rate Shifting

Maskformer with Improved Encoder-Decoder Module for Semantic Segmentation of Fine-Resolution Remote Sensing Images.

MP-Former: Mask-Piloted Transformer for Image Segmentation

Learning with Unmasked Tokens Drives Stronger Vision Learners