Abstract:Detecting human-object interactions (HOIs) is an intricate challenge in the field of computer vision. Existing methods for HOI detection heavily rely on appearance-based features, but these may not fully capture all the essential characteristics necessary for accurate detection. To overcome these challenges, we propose an innovative graph-based approach called TMGHOI (Translational Model for Human-Object Interaction Detection). Our method effectively captures the sentiment representation of HOIs by integrating both spatial and semantic knowledge. By representing HOIs as a graph, where the interaction components serve as nodes and their spatial relationships as edges. To extract crucial spatial and semantic information, TMGHOI employs separate spatial and semantic encoders. Subsequently, these encodings are combined to construct a knowledge graph that effectively captures the sentiment representation of HOIs. Additionally, the ability to incorporate prior knowledge enhances the understanding of interactions, further boosting detection accuracy. We conducted extensive evaluations on the widely-used HICO-DET datasets to demonstrate the effectiveness of TMGHOI. Our approach outperformed existing state-of-the-art graph-based methods by a significant margin, showcasing its potential as a superior solution for HOI detection. We are confident that TMGHOI has the potential to significantly improve the accuracy and efficiency of HOI detection. Its integration of spatial and semantic knowledge, along with its computational efficiency and practicality, makes it a valuable tool for researchers and practitioners in the computer vision community. As with any research, we acknowledge the importance of further exploration and evaluation on various datasets to establish the generalizability and robustness of our proposed method.

Modeling 4D Human-Object Interactions for Joint Event Segmentation, Recognition, and Object Localization.

Modeling 4d Human-Object Interactions for Event and Object Recognition

Detecting Human-Object Interactions with Object-Guided Cross-Modal Calibrated Semantics.

In-Hand 3D Object Reconstruction from a Monocular RGB Video

HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction

UniHOI: Learning Fast, Dense and Generalizable 4D Reconstruction for Egocentric Hand Object Interaction Videos

Hi4D: 4D Instance Segmentation of Close Human Interaction

Detailed 2D-3D Joint Representation for Human-Object Interaction

Learning Human-Object Interaction via Interactive Semantic Reasoning

Human-object Interaction Detection with Depth-Augmented Clues

Kinematics-based 3D Human-Object Interaction Reconstruction from Single View

HiSC4D: Human-centered interaction and 4D Scene Capture in Large-scale Space Using Wearable IMUs and LiDAR

ST-HOI: A Spatial-Temporal Baseline for Human-Object Interaction Detection in Videos

Detecting Human-Object Interactions in Videos by Modeling the Trajectory of Objects and Human Skeleton

F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactions

HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environment

HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects

Discovering A Variety of Objects in Spatio-Temporal Human-Object Interactions

Recognising Human-Object Interaction Via Exemplar Based Modelling

TMHOI: Translational Model for Human-Object Interaction Detection

Hierarchical HOI Detection Framework Augmented by Human Interactive Intention