Abstract:People’s interactions with each other form the social relations in society. Understanding human social relations in the public space is of great importance for supporting the public administrations. Recognizing social relations through visual data captured by remote sensing cameras is one of the most efficient ways to observe human interactions in a public space. Generally speaking, persons in the same scene tend to know each other, and the relations between person pairs are strongly correlated. The scene information in which people interact is also one of the important cues for social relation recognition (SRR). The existing works have not explored the correlations between the scene information and people’s interactions. The scene information has only been extracted on a simple level and high level semantic features to support social relation understanding are lacking. To address this issue, we propose a social relation structure-aware local–global model for SRR to exploit the high-level semantic global information of the scene where the social relation structure is explored. In our proposed model, the graph neural networks (GNNs) are employed to reason through the interactions (local information) between social relations and the global contextual information contained in the constructed scene-relation graph. Experiments demonstrate that our proposed local–global information-reasoned social relation recognition model (SRR-LGR) can reason through the local–global information. Further, the results of the final model show that our method outperforms the state-of-the-art methods. In addition, we have further discussed whether the global information contributes equally to different social relations in the same scene, by exploiting an attention mechanism in our proposed model. Further applications of SRR for human-observation are also exploited.

HF-SRGR: a New Hybrid Feature-Driven Social Relation Graph Reasoning Model

Progressive Graph Reasoning-Based Social Relation Recognition

Graph-Based Social Relation Reasoning

Multi-Granularity Reasoning for Social Relation Recognition From Images

SRR-LGR: Local–Global Information-Reasoned Social Relation Recognition for Human-Oriented Observation

Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph

Principal Relation Component Reasoning-Enhanced Social Relation Recognition

Deep Reasoning with Knowledge Graph for Social Relationship Understanding

Multi-Level Transformer-Based Social Relation Recognition

Social Relation Reasoning Based on Triangular Constraints

Multi-scale Features Based Interpersonal Relation Recognition Using Higher-Order Graph Neural Network.

Learning Social Spatio-Temporal Relation Graph in the Wild and a Video Benchmark.

Social Relation Analysis from Videos Via Multi-entity Reasoning

Social Relation Recognition from Videos Via Multi-Scale Spatial-Temporal Reasoning

Multi-Graph Heterogeneous Interaction Fusion for Social Recommendation

Enhancing Social Relation Inference with Concise Interaction Graph and Discriminative Scene Representation

Linking People Across Text and Images Based on Social Relation Reasoning.

Hypergraph-Enhanced Self-Supervised Robust Graph Learning for Social Recommendation.

Unveiling Social Relations: Leveraging Interpersonal Similarity Learning for Social Relation Recognition.

Hybrid-attention Mechanism Based Heterogeneous Graph Representation Learning

Graph-based High-Order Relation Discovery for Fine-grained Recognition