Abstract:Human-Object Interaction (HOI), as an important problem in computer vision, requires locating the human-object pair and identifying the interactive relationships between them. The HOI instance has a greater span in spatial, scale, and task than the individual object instance, making its detection more susceptible to noisy backgrounds. To alleviate the disturbance of noisy backgrounds on HOI detection, it is necessary to consider the input image information to generate fine-grained anchors which are then leveraged to guide the detection of HOI instances. However, it has the following challenges. i) how to extract pivotal features from the images with complex background information is still an open question. ii) how to semantically align the extracted features and query embeddings is also a difficult issue. In this paper, a novel end-to-end transformer-based framework (FGAHOI) is proposed to alleviate the above problems. FGAHOI comprises three dedicated components namely, multi-scale sampling (MSS), hierarchical spatial-aware merging (HSAM) and task-aware merging mechanism (TAM). MSS extracts features of humans, objects and interaction areas from noisy backgrounds for HOI instances of various scales. HSAM and TAM semantically align and merge the extracted features and query embeddings in the hierarchical spatial and task perspectives in turn. In the meanwhile, a novel training strategy Stage-wise Training Strategy is designed to reduce the training pressure caused by overly complex tasks done by FGAHOI. In addition, we propose two ways to measure the difficulty of HOI detection and a novel dataset, i.e., HOI-SDC for the two challenges (Uneven Distributed Area in Human-Object Pairs and Long Distance Visual Modeling of Human-Object Pairs) of HOI instances detection. Experiments are conducted on three benchmarks: HICO-DET, HOI-SDC and V-COCO. Our model outperforms the state-of-the-art HOI detection methods, and the extensive ablations reveal the merits of our proposed contribution.

Turbo Learning Framework for Human-Object Interactions Recognition and Human Pose Estimation

Exploring Pose-Aware Human-Object Interaction Via Hybrid Learning

Learning Human-Object Interaction Detection Using Interaction Points

Exploiting Human Pose and Scene Information for Interaction Detection

Cascaded Parsing of Human-Object Interaction Recognition

Cascaded Human-Object Interaction Recognition

Reformulating HOI Detection as Adaptive Set Prediction

ViPLO: Vision Transformer based Pose-Conditioned Self-Loop Graph for Human-Object Interaction Detection

FGAHOI: Fine-Grained Anchors for Human-Object Interaction Detection.

Learning Transferable Human-Object Interaction Detector with Natural Language Supervision

From Category to Scenery: An End-to-End Framework for Multi-Person Human-Object Interaction Recognition in Videos

CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics

Learning Human-Object Interaction via Interactive Semantic Reasoning

Simultaneous Learning from Human Pose and Object Cues for Real-Time Activity Recognition

Disentangled Interaction Representation for One-Stage Human-Object Interaction Detection

UAHOI: Uncertainty-aware Robust Interaction Learning for HOI Detection

Parallel disentangling network for human–object interaction detection

Point-Based Learnable Query Generator for Human–Object Interaction Detection

TMHOI: Translational Model for Human-Object Interaction Detection

Learning to Detect Human-Object Interactions

ContextHOI: Spatial Context Learning for Human-Object Interaction Detection