Abstract:Weakly supervised semantic segmentation is receiving great attention due to its low human annotation cost. In this paper, we aim to tackle bounding box supervised semantic segmentation, i.e., training accurate semantic segmentation models using bounding box annotations as supervision. To this end, we propose Affinity Attention Graph Neural Network ($A^2$GNN). Following previous practices, we first generate pseudo semantic-aware seeds, which are then formed into semantic graphs based on our newly proposed affinity Convolutional Neural Network (CNN). Then the built graphs are input to our $A^2$GNN, in which an affinity attention layer is designed to acquire the short- and long- distance information from soft graph edges to accurately propagate semantic labels from the confident seeds to the unlabeled pixels. However, to guarantee the precision of the seeds, we only adopt a limited number of confident pixel seed labels for $A^2$GNN, which may lead to insufficient supervision for training. To alleviate this issue, we further introduce a new loss function and a consistency-checking mechanism to leverage the bounding box constraint, so that more reliable guidance can be included for the model optimization. Experiments show that our approach achieves new state-of-the-art performances on Pascal VOC 2012 datasets (val: 76.5\%, test: 75.2\%). More importantly, our approach can be readily applied to bounding box supervised instance segmentation task or other weakly supervised semantic segmentation tasks, with state-of-the-art or comparable performance among almot all weakly supervised tasks on PASCAL VOC or COCO dataset. Our source code will be available at <a class="link-external link-https" href="https://github.com/zbf1991/A2GNN" rel="external noopener nofollow">this https URL</a>.

Affinity Derivation for Accurate Instance Segmentation.

Affinity Derivation and Graph Merge for Instance Segmentation.

Learning to Model Pixel-Embedded Affinity for Homogeneous Instance Segmentation.

Deep Affinity Net: Instance Segmentation via Affinity

Affinity Fusion Graph-Based Framework for Natural Image Segmentation

Label-efficient Segmentation via Affinity Propagation

SSAP: Single-Shot Instance Segmentation With Affinity Pyramid

Improving Semantic Segmentation via Dilated Affinity

Affinity3D: Propagating Instance-Level Semantic Affinity for Zero-Shot Point Cloud Semantic Segmentation

An Adaptive Affinity Graph with Subspace Pursuit for Natural Image Segmentation

Learning Hybrid Semantic Affinity for Point Cloud Segmentation

Adaptive Affinity Loss and Erroneous Pseudo-Label Refinement for Weakly Supervised Semantic Segmentation

Associate Semantic-Instance Segmentation of 3D Point Clouds Based on Local Feature Extraction

SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation

Affinity Space Adaptation for Semantic Segmentation Across Domains

Learning Inter-Superpoint Affinity for Weakly Supervised 3D Instance Segmentation

Class-Specific Affinity based Weakly Supervised Semantic Segmentation with Neutral Region Exploration

Affinity Attention Graph Neural Network for Weakly Supervised Semantic Segmentation

Unsupervised Co-Segmentation of 3D Shapes Via Affinity Aggregation Spectral Clustering.

Texture Image Segmentation Using Affinity Propagation And Spectral Clustering

Pixel and region level information fusion in membership regularized fuzzy clustering for image segmentation