Abstract:Weakly supervised semantic segmentation (WSSS) models relying on class activation maps (CAMs) have achieved desirable performance comparing to the non-CAMs-based counterparts. However, to guarantee WSSS task feasible, we need to generate pseudo labels by expanding the seeds from CAMs which is complex and time-consuming, thus hindering the design of efficient end-to-end (single-stage) WSSS approaches. To tackle the above dilemma, we resort to the off-the-shelf and readily accessible saliency maps for directly obtaining pseudo labels given the image-level class labels. Nevertheless, the salient regions may contain noisy labels and cannot seamlessly fit the target objects, and saliency maps can only be approximated as pseudo labels for simple images containing single-class objects. As such, the achieved segmentation model with these simple images cannot generalize well to the complex images containing multi-class objects. To this end, we propose an end-to-end multi-granularity denoising and bidirectional alignment (MDBA) model, to alleviate the noisy label and multi-class generalization issues. Specifically, we propose the online noise filtering and progressive noise detection modules to tackle image-level and pixel-level noise, respectively. Moreover, a bidirectional alignment mechanism is proposed to reduce the data distribution gap at both input and output space with simple-to-complex image synthesis and complex-to-simple adversarial learning. MDBA can reach the mIoU of 69.5% and 70.2% on validation and test sets for the PASCAL VOC 2012 dataset. The source codes and models have been made available at https://github.com/NUST-MachineIntelligence-Laboratory/MDBA.

Noisy Tag Alignment with Image Regions.

Social tag alignment with image regions by sparse reconstructions.

TagAlign: Improving Vision-Language Alignment with Multi-Tag Classification

Towards More Precise Social Image-Tag Alignment

Tag Localization with Spatial Correlations and Joint Group Sparsity

Image Tagging with Social Assistance

Image region label refinement using spatial position relation graph.

Local Image Tagging Via Graph Regularized Joint Group Sparsity

A New Fine-grained Alignment Method for Image-text Matching

DAC: 2D-3D Retrieval with Noisy Labels via Divide-and-Conquer Alignment and Correction

A Locality Sensitive Low-Rank Model for Image Tag Completion

ProAlignNet : Unsupervised Learning for Progressively Aligning Noisy Contours

Image Region Labeling by Exploring Contextual Information of Visual Spatial and Semantic Concepts

Exploiting Spatial Context Constraints for Automatic Image Region Annotation.

Automatic tag-to-region assignment via multiple instance learning

Image Region Annotation Based on Segmentation and Semantic Correlation Analysis

Graph Alignment with Noisy Supervision

Multi-Granularity Denoising and Bidirectional Alignment for Weakly Supervised Semantic Segmentation

Learning to Segment from Noisy Annotations: A Spatial Correction Approach

Object Co-Segmentation Via Salient and Common Regions Discovery

Nonparametric Label-to-Region by search