Abstract:Understanding human interaction with objects is an important research topic for embodied Artificial Intelligence and identifying the objects that humans are interacting with is a primary problem for interaction understanding. Existing methods rely on frame-based detectors to locate interacting objects. However, this approach is subjected to heavy occlusions, background clutter, and distracting objects. To address the limitations, in this paper, we propose to leverage spatio-temporal information of hand-object interaction to track interactive objects under these challenging cases. Without prior knowledge of the general objects to be tracked like object tracking problems, we first utilize the spatial relation between hands and objects to adaptively discover the interacting objects from the scene. Second, the consistency and continuity of the appearance of objects between successive frames are exploited to track the objects. With this tracking formulation, our method also benefits from training on large-scale general object-tracking datasets. We further curate a video-level hand-object interaction dataset for testing and evaluation from 100DOH. The quantitative results demonstrate that our proposed method outperforms the state-of-the-art methods. Specifically, in scenes with continuous interaction with different objects, we achieve an impressive improvement of about 10% as evaluated using the Average Precision (AP) metric. Our qualitative findings also illustrate that our method can produce more continuous trajectories for interacting objects.

Inter-Person Occlusion Handling with Social Interaction for Online Multi-Pedestrian Tracking.

Humans Tracking In The Complicated Background By Multi-Cue Integration

Tracking Multiple People under Occlusion and Across Cameras Using Probabilistic Models

OccluTrack: Rethinking Awareness of Occlusion for Enhancing Multiple Pedestrian Tracking

Multi-person Multi-Camera Tracking for Live Stream Videos Based on Improved Motion Model and Matching Cascade

An anti-occlusion optimization algorithm for multiple pedestrian tracking

OSMO: Online Specific Models for Occlusion in Multiple Object Tracking under Surveillance Scene.

Towards Occlusion Handling: Object Tracking with Background Estimation.

AHOR: Online Multi-Object Tracking With Authenticity Hierarchizing and Occlusion Recovery

Robust Occlusion Handling in Object Tracking

Occlusion Handling Based on Motion Estimation for Multi-Object Tracking

InterTracker: Discovering and Tracking General Objects Interacting with Hands in the Wild

Multiple Human Tracking Based on Multi-view Upper-Body Detection and Discriminative Learning

Online multi-object tracking with pedestrian re-identification and occlusion processing

Effective multiple pedestrian tracking system in video surveillance with monocular stationary camera

Shape and Pose Estimation for Closely Interacting Persons Using Multi-view Images.

A Boosted Multi-Task Model for Pedestrian Detection with Occlusion Handling.

Tracking Occluded Objects And Recovering Incomplete Trajectories By Reasoning About Containment Relations And Human Actions

Tracking as A Whole: Multi-Target Tracking by Modeling Group Behavior with Sequential Detection

Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking

UNOC: Understanding Occlusion for Embodied Presence in Virtual Reality