Abstract:Scene graph generation aims to detect visual entities and relationships between them from an image. The object-level visual information is of vital importance for predicting accurate relationships. However, most existing methods essentially encode visual information with coarse supervised information, since they regard different relationships as mutually exclusive semantics with equal-distance labels by taking cross-entropy function as the main training loss. Intuitively, different relationship semantics naturally have their own similarity and dissimilarity with different level distances, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , the topological information of relationship semantics. It can serve as an inspiring hint to aid learning to grasp the key related visual information. Accordingly, we propose a Semantically Similarity-wise Dual-branch Network (SSDN) which introduces topological information of relationship semantics as extra supervision to aid learning extracting and encoding relationship-related visual information. To avoid possible chaotic feature learning and enable the introduced knowledge to be better absorbed during inference, we design a dual-branch framework consisting of an auxiliary branch and an inference branch. The topological information extracted from the groundtruth is introduced at the front end of the auxiliary branch which then generates a soft embedding to be propagated to the inference branch in a knowledge distillation manner. Extensive experiments show that our model averagely outperforms state-of-the-art approaches on benchmark Visual Genome and VRD significantly, which demonstrates its effectiveness and superiority.

DH-GCN: Saliency-Aware Complex Scene Graph Generation Using Dual-Hierarchy Graph Convolutional Network

Scene Graph Generation With Hierarchical Context

Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledge

Attention Redirection Transformer with Semantic Oriented Learning for Unbiased Scene Graph Generation

Semantically Similarity-Wise Dual-Branch Network for Scene Graph Generation

Relationship-Aware Primal-Dual Graph Attention Network For Scene Graph Generation.

Hyper-relationship Learning Network for Scene Graph Generation

Multi-Scale Graph Attention Network for Scene Graph Generation

Salient Object Detection Based on Unified Graph Neural Network Joint Learning

Scene Graph Generation using Depth-based Multimodal Network.

Dual Graph Convolutional Network for Semantic Segmentation.

Scene Graph Generation Via Convolutional Message Passing and Class-Aware Memory Embeddings

Attentive Gated Graph Neural Network for Image Scene Graph Generation

DAGCN: Dynamic and Adaptive Graph Convolutional Network for Salient Object Detection.

Attention LSTM for Scene Graph Generation

Graph R-CNN for Scene Graph Generation

Explore Hierarchical Relations Reasoning and Global Information Aggregation

Scene Graph Disentanglement and Composition for Generalizable Complex Image Generation

Scene Graph Generation Via Multi-Relation Classification and Cross-Modal Attention Coordinator.

Exploiting Relationship for Complex-scene Image Generation

Unconditional Scene Graph Generation