Abstract:Cameras are essential vision instruments to capture images for pattern detection and measurement. Human-object interaction (HOI) detection is one of the most popular pattern detection approaches for captured human-centric visual scenes. Recently, Transformer-based models have become the dominant approach for HOI detection due to their advanced network architectures and thus promising results. However, most of them follow the one-stage design of vanilla Transformer, leaving rich geometric priors under-exploited and leading to compromised performance especially when occlusion occurs. Given that geometric features tend to outperform visual ones in occluded scenarios and offer information that complements visual cues, we propose a novel end-to-end Transformer-style HOI detection model, i.e., geometric features enhanced HOI detector (GeoHOI). One key part of the model is a new unified self-supervised keypoint learning method named UniPointNet that bridges the gap of consistent keypoint representation across diverse object categories, including humans. GeoHOI effectively upgrades a Transformer-based HOI detector benefiting from the keypoints similarities measuring the likelihood of human-object interactions as well as local keypoint patches to enhance interaction query representation, so as to boost HOI predictions. Extensive experiments show that the proposed method outperforms the state-of-the-art models on V-COCO and achieves competitive performance on HICO-DET. Case study results on the post-disaster rescue with vision-based instruments showcase the applicability of the proposed GeoHOI in real-world applications.

Focus and Adjust: Progressive Refinement Network for Human Object Interaction Detection

Parallel Reasoning Network for Human-Object Interaction Detection

A Transformer-Based Object Detector with Coarse-Fine Crossing Representations

A Novel Part Refinement Tandem Transformer for Human-Object Interaction Detection

Category-Aware Transformer Network for Better Human-Object Interaction Detection

Reformulating HOI Detection as Adaptive Set Prediction

Parallel disentangling network for human–object interaction detection

End-to-End Human Object Interaction Detection with HOI Transformer

Mask-Guided Transformer for Human-Object Interaction Detection

Human–object interaction detection based on disentangled axial attention transformer

Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detection

Multi-Scale Human-Object Interaction Detector.

Geometric Features Enhanced Human-Object Interaction Detection

Geometric Features Enhanced Human–Object Interaction Detection

An Improved Human-Object Interaction Detection Network

Cross-Level Guided Attention for Human-Object Interaction Detection

Human Object Interaction Detection Based on Feature Optimization and Key Human-Object Enhancement.

A Transformer-based Dual Position Attention Network for Recognizing Human-object Interaction

Hierarchical Reasoning Network for Human-Object Interaction Detection

RR-Net: Relation Reasoning for End-to-End Human-Object Interaction Detection

GTNet:Guided Transformer Network for Detecting Human-Object Interactions