Abstract:Weakly supervised semantic segmentation is receiving great attention due to its low human annotation cost. In this paper, we aim to tackle bounding box supervised semantic segmentation, i.e., training accurate semantic segmentation models using bounding box annotations as supervision. To this end, we propose Affinity Attention Graph Neural Network ($A^2$GNN). Following previous practices, we first generate pseudo semantic-aware seeds, which are then formed into semantic graphs based on our newly proposed affinity Convolutional Neural Network (CNN). Then the built graphs are input to our $A^2$GNN, in which an affinity attention layer is designed to acquire the short- and long- distance information from soft graph edges to accurately propagate semantic labels from the confident seeds to the unlabeled pixels. However, to guarantee the precision of the seeds, we only adopt a limited number of confident pixel seed labels for $A^2$GNN, which may lead to insufficient supervision for training. To alleviate this issue, we further introduce a new loss function and a consistency-checking mechanism to leverage the bounding box constraint, so that more reliable guidance can be included for the model optimization. Experiments show that our approach achieves new state-of-the-art performances on Pascal VOC 2012 datasets (val: 76.5\%, test: 75.2\%). More importantly, our approach can be readily applied to bounding box supervised instance segmentation task or other weakly supervised semantic segmentation tasks, with state-of-the-art or comparable performance among almot all weakly supervised tasks on PASCAL VOC or COCO dataset. Our source code will be available at <a class="link-external link-https" href="https://github.com/zbf1991/A2GNN" rel="external noopener nofollow">this https URL</a>.

Affinity Derivation and Graph Merge for Instance Segmentation.

Affinity Derivation for Accurate Instance Segmentation.

Deep Affinity Net: Instance Segmentation via Affinity

Learning to Model Pixel-Embedded Affinity for Homogeneous Instance Segmentation.

Affinity Fusion Graph-Based Framework for Natural Image Segmentation

Associate Semantic-Instance Segmentation of 3D Point Clouds Based on Local Feature Extraction

SSAP: Single-Shot Instance Segmentation With Affinity Pyramid

An Adaptive Affinity Graph with Subspace Pursuit for Natural Image Segmentation

SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation

Biological Instance Segmentation with a Superpixel-Guided Graph.

Affinity3D: Propagating Instance-Level Semantic Affinity for Zero-Shot Point Cloud Semantic Segmentation

Label-efficient Segmentation via Affinity Propagation

Improving Semantic Segmentation via Dilated Affinity

Joint Point Clouds Semantic and Instance Segmentation by Local Aggregation and Clustering

Research on Attention-Based Multiscale Information Fusion with the Real-Time Instance Segmentation Method

Semantic Segmentation Using Multiple Graphs With Block-Diagonal Constraints

Learning Hybrid Semantic Affinity for Point Cloud Segmentation

Affinity Attention Graph Neural Network for Weakly Supervised Semantic Segmentation

Semantic Segmentation of Point Cloud Scene via Multi-Scale Feature Aggregation and Adaptive Fusion

Class-Specific Affinity based Weakly Supervised Semantic Segmentation with Neutral Region Exploration

Associating Inter-Image Salient Instances For Weakly Supervised Semantic Segmentation