Abstract:Scene graph generation refers to the task of identifying the objects and specifically the relationships between the objects from an image. Existing scene graph generation methods generally use the bounding boxes region features of objects to identify the relationships between objects. However, we feel that the overlap region features of two objects may play an important role in fine‐grained relationship identification. In fact, some fine‐grained relationships can only be obtained from the overlap region features of two objects. Therefore, we propose the Multi‐Branch Feature Combination (MFC) module and Overlap Region Transformer (ORT) module to comprehensively obtain the visual features contained in the overlap regions of two objects. Concretely, the MFC module uses deconvolution and multi‐branch dilation convolution to obtain high‐pixels and multi‐receptive field features in the overlap regions. The ORT module uses the vision transformer to obtain the self‐attention of the overlap regions. The joint use of these two modules achieves the mutual complementation of local connectivity properties of convolution and the global connectivity properties of attention. We also design a Geometrical Center Augmented (GCA) module to obtain the relative position information of the geometric centers between two objects, to prevent the problem that only relying on the scale of the overlap region cannot accurately capture the relationship between two objects. Experiments show that our model ORGC (Overlap Region and Geometrical Center), the combination of the MFC module, the ORT module, and the GCA module, can enhance the performance of fine‐grained relation identification. On the Visual Genome dataset, our model outperforms the current state‐of‐the‐art model by 4.4% on the R@50 evaluation metric, reaching a state‐of‐the‐art result of 33.88.

Large-scale Outdoor Scene Classification by Boosting a Set of Highly Discriminative and Low Redundant Graphlets

Compositional Boosting for Computing Hierarchical Image Structures

Learning a Probabilistic Topology Discovering Model for Scene Categorization.

Discovering Discriminative Graphlets for Aerial Image Categories Recognition

AUG: A New Dataset and An Efficient Model for Aerial Image Urban Scene Graph Generation

OpenGraph: Open-Vocabulary Hierarchical 3D Graph Representation in Large-Scale Outdoor Environments

Scene Classification Based on Heterogeneous Features of Multi-Source Data

Massive-Scale Visual Information Retrieval towards City Residential Environment Surveillance

Large-Scale Aerial Image Categorization by Multi-Task Discriminative Topologies Discovery.

Graph-Based Embedding Smoothing Network for Few-Shot Scene Classification of Remote Sensing Images

Fine‐Grained Scene Graph Generation with Overlap Region and Geometrical Center

Recognition of Indoor Scenes Using 3-D Scene Graphs

Deep social grouping network for large scenes with multiple subjects

Deeply Encoding Stable Patterns From Contaminated Data for Scenery Image Recognition

Inter-object Discriminative Graph Modeling for Indoor Scene Recognition

MLMG-SGG: Multilabel Scene Graph Generation with Multigrained Features.

Scalable Scene Understanding Via Saliency Consensus

Adaptive Graph Modeling With Self-Training for Heterogeneous Cross-Scene Hyperspectral Image Classification

MultiScene: A Large-scale Dataset and Benchmark for Multi-scene Recognition in Single Aerial Images

A novel approach based on cluster-group for classification of 3D residential scene

Satellite Video Multi-Label Scene Classification With Spatial and Temporal Feature Cooperative Encoding: A Benchmark Dataset and Method