Abstract:Recognizing how objects interact with each other is a crucial task in visual recognition. If we define the context of the interaction to be the objects involved, then most current methods can be categorized as either: (i) training a single classifier on the combination of the interaction and its context; or (ii) aiming to recognize the interaction independently of its explicit context. Both methods suffer limitations: the former scales poorly with the number of combinations and fails to generalize to unseen combinations, while the latter often leads to poor interaction recognition performance due to the difficulty of designing a context-independent interaction classifier.To mitigate those drawbacks, this paper proposes an alternative, context-aware interaction recognition framework. The key to our method is to explicitly construct an interaction classifier which combines the context, and the interaction. The context is encoded via word2vec into a semantic space, and is used to derive a classification result for the interaction. The proposed method still builds one classifier for one interaction (as per type (ii) above), but the classifier built is adaptive to context via weights which are context dependent. The benefit of using the semantic space is that it naturally leads to zero-shot generalizations in which semantically similar contexts (subject-object pairs) can be recognized as suitable contexts for an interaction, even if they were not observed in the training set. Our method also scales with the number of interaction-context pairs since our model parameters do not increase with the number of interactions. Thus our method avoids the limitation of both approaches. We demonstrate experimentally that the proposed framework leads to improved performance for all investigated interaction representations and datasets.

Deep Contextual Attention for Human-Object Interaction Detection

Detecting Human-Object Interactions with Object-Guided Cross-Modal Calibrated Semantics.

ContextHOI: Spatial Context Learning for Human-Object Interaction Detection

Learning Human-Object Interaction Detection Using Interaction Points

Human-object Interaction Detection with Depth-Augmented Clues

Human Object Interaction Detection using Two-Direction Spatial Enhancement and Exclusive Object Prior

Effective Actor-centric Human-object Interaction Detection

Visual Compositional Learning for Human-Object Interaction Detection

Graph-based Method for Human-Object Interactions Detection

Human Object Interaction Detection via Multi-level Conditioned Network

Spatially-Aware Human-Object Interaction Detection with Cross-Modal Enhancement

A Robust Context Attention Network for Human Hand Detection

Action-Guided Attention Mining and Relation Reasoning Network for Human-Object Interaction Detection

Attentive Contexts for Object Detection

Distance Matters in Human-Object Interaction Detection

Learning Self- and Cross-Triplet Context Clues for Human-Object Interaction Detection

Hierarchical HOI Detection Framework Augmented by Human Interactive Intention

An Improved Human-Object Interaction Detection Network

Improving 3D Object Detection with Context-Aware and Dimensional Interaction Attention

Towards Context-Aware Interaction Recognition for Visual Relationship Detection

Human-Object Interaction Detection via Weak Supervision