Abstract:Detecting human-object interactions (HOIs) is an intricate challenge in the field of computer vision. Existing methods for HOI detection heavily rely on appearance-based features, but these may not fully capture all the essential characteristics necessary for accurate detection. To overcome these challenges, we propose an innovative graph-based approach called TMGHOI (Translational Model for Human-Object Interaction Detection). Our method effectively captures the sentiment representation of HOIs by integrating both spatial and semantic knowledge. By representing HOIs as a graph, where the interaction components serve as nodes and their spatial relationships as edges. To extract crucial spatial and semantic information, TMGHOI employs separate spatial and semantic encoders. Subsequently, these encodings are combined to construct a knowledge graph that effectively captures the sentiment representation of HOIs. Additionally, the ability to incorporate prior knowledge enhances the understanding of interactions, further boosting detection accuracy. We conducted extensive evaluations on the widely-used HICO-DET datasets to demonstrate the effectiveness of TMGHOI. Our approach outperformed existing state-of-the-art graph-based methods by a significant margin, showcasing its potential as a superior solution for HOI detection. We are confident that TMGHOI has the potential to significantly improve the accuracy and efficiency of HOI detection. Its integration of spatial and semantic knowledge, along with its computational efficiency and practicality, makes it a valuable tool for researchers and practitioners in the computer vision community. As with any research, we acknowledge the importance of further exploration and evaluation on various datasets to establish the generalizability and robustness of our proposed method.

Spatially-Aware Human-Object Interaction Detection with Cross-Modal Enhancement

Detecting Human-Object Interactions with Object-Guided Cross-Modal Calibrated Semantics.

Human Object Interaction Detection using Two-Direction Spatial Enhancement and Exclusive Object Prior

In vivo vascular freezing in clinical microvascular transfer

Exploring Pose-Aware Human-Object Interaction Via Hybrid Learning

ContextHOI: Spatial Context Learning for Human-Object Interaction Detection

Distance Matters in Human-Object Interaction Detection

Re-mine, Learn and Reason: Exploring the Cross-modal Semantic Correlations for Language-guided HOI detection

Open-World Human-Object Interaction Detection via Multi-modal Prompts

Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Models

Efficient Adaptive Human-Object Interaction Detection with Concept-guided Memory

TMHOI: Translational Model for Human-Object Interaction Detection

Effective Actor-centric Human-object Interaction Detection

Toward Open-Set Human Object Interaction Detection

Amplifying Key Cues for Human-Object-Interaction Detection

Learning Transferable Human-Object Interaction Detector with Natural Language Supervision

Exploring Interactive Semantic Alignment for Efficient HOI Detection with Vision-language Model

Learning Human-Object Interaction Detection Using Interaction Points

HODN: Disentangling Human-Object Feature for HOI Detection

Generating Human-Centric Visual Cues for Human-Object Interaction Detection via Large Vision-Language Models

Enhancing HOI Detection with Contextual Cues from Large Vision-Language Models