Abstract:Detecting human-object interactions (HOIs) is an intricate challenge in the field of computer vision. Existing methods for HOI detection heavily rely on appearance-based features, but these may not fully capture all the essential characteristics necessary for accurate detection. To overcome these challenges, we propose an innovative graph-based approach called TMGHOI (Translational Model for Human-Object Interaction Detection). Our method effectively captures the sentiment representation of HOIs by integrating both spatial and semantic knowledge. By representing HOIs as a graph, where the interaction components serve as nodes and their spatial relationships as edges. To extract crucial spatial and semantic information, TMGHOI employs separate spatial and semantic encoders. Subsequently, these encodings are combined to construct a knowledge graph that effectively captures the sentiment representation of HOIs. Additionally, the ability to incorporate prior knowledge enhances the understanding of interactions, further boosting detection accuracy. We conducted extensive evaluations on the widely-used HICO-DET datasets to demonstrate the effectiveness of TMGHOI. Our approach outperformed existing state-of-the-art graph-based methods by a significant margin, showcasing its potential as a superior solution for HOI detection. We are confident that TMGHOI has the potential to significantly improve the accuracy and efficiency of HOI detection. Its integration of spatial and semantic knowledge, along with its computational efficiency and practicality, makes it a valuable tool for researchers and practitioners in the computer vision community. As with any research, we acknowledge the importance of further exploration and evaluation on various datasets to establish the generalizability and robustness of our proposed method.

Detecting Zero-Shot Human-Object Interaction with Visual-Text Modeling

Detecting Human-Object Interactions with Object-Guided Cross-Modal Calibrated Semantics.

Towards Zero-shot Human-Object Interaction Detection via Vision-Language Integration

Toward Open-Set Human Object Interaction Detection

Exploring Interactive Semantic Alignment for Efficient HOI Detection with Vision-language Model

Zero-Shot Human-Object Interaction Detection via Similarity Propagation

End-to-End Zero-Shot HOI Detection Via Vision and Language Knowledge Distillation

Visual Compositional Learning for Human-Object Interaction Detection

TMHOI: Translational Model for Human-Object Interaction Detection

Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Models

Learning Transferable Human-Object Interaction Detector with Natural Language Supervision

Improving Human-Object Interaction Detection via Virtual Image Learning

Learning Human-Object Interaction Detection Using Interaction Points

EZ-HOI: VLM Adaptation via Guided Prompt Learning for Zero-Shot HOI Detection

CLIP4HOI: Towards Adapting CLIP for Practical Zero-Shot HOI Detection.

Exploring Pose-Aware Human-Object Interaction Via Hybrid Learning

A Review of Human-Object Interaction Detection

HODN: Disentangling Human-Object Feature for HOI Detection

In vivo vascular freezing in clinical microvascular transfer

Generating Human-Centric Visual Cues for Human-Object Interaction Detection via Large Vision-Language Models