Abstract:Intelligently tracking objects with varied shapes, color, lighting conditions, and backgrounds is an extremely useful application in many HCI applications, such as human body motion capture, hand gesture recognition, and virtual reality (VR) games. However, accurately tracking different objects under uncontrolled environments is a tough challenge due to the possibly dynamic object parts, varied lighting conditions, and sophisticated backgrounds. In this work, we propose a novel semantically-aware object tracking framework, wherein the key is weakly-supervised learning paradigm that optimally transfers the video-level semantic tags into various regions. More specifically, give a set of training video clips, each of which is associated with multiple video-level semantic tags, we first propose a weakly-supervised learning algorithm to transfer the semantic tags into various video regions. The key is a MIL (Zhong et al., 2020) [1]-based manifold embedding algorithm that maps the entire video regions into a semantic space, wherein the video-level semantic tags are well encoded. Afterward, for each video region, we use the semantic feature combined with the appearance feature as its representation. We designed a multi-view learning algorithm to optimally fuse the above two types of features. Based on the fused feature, we learn a probabilistic Gaussian mixture model to predict the target probability of each candidate window, where the window with the maximal probability is output as the tracking result. Comprehensive comparative results on a challenging pedestrian tracking task as well as the human hand gesture recognition have demonstrated the effectiveness of our method. Moreover, visualized tracking results have shown that non-rigid objects with moderate occlusions can be well localized by our method.

Multi-Object Tracking Based on Semantic Information for Autonomous Vehicle

Visual Tracking Based on Semantic and Similarity Learning

Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Driving

Semantically Enhanced Multi-Object Detection and Tracking for Autonomous Vehicles.

A Tracking-By-Detection Based 3D Multiple Object Tracking for Autonomous Driving

Three-Dimensional Multi-Object Tracking Based on Feature Fusion and Similarity Estimation Network

Multi-object tracking via deep feature fusion and association analysis

Multi-object Tracking for Autonomous Driving with a Monocular Camera Subject to Missing Data

T2Track: Multi-Object Tracking by Associating Between Tracks

Online Multi-Object Tracking Using Joint Domain Information in Traffic Scenarios

Robust Multi-Modality Multi-Object Tracking

One-shot Multi-Object Tracking Using CNN-based Networks with Spatial-Channel Attention Mechanism

3D Multi-object Tracking in Autonomous Driving: A Survey

Spatial-Attention Location-Aware Multi-Object Tracking

A Multi-Object Tracking Approach Combining Contextual Features and Trajectory Prediction

Multi-Object Tracking Algorithm for RGB-D Images Based on Asymmetric Dual Siamese Networks

Multiple Instance Deep Learning for Weakly-Supervised Visual Object Tracking

A Dynamic 3D Multi-Object Tracking Method Based on Spatiotemporal Features

Spatial-Semantic and Temporal Attention Mechanism-Based Online Multi-Object Tracking

Multi-3D-Object Tracking by Fusing RGB and 3D-LiDAR Data

TEMI-MOT: Towards Efficient Multi-Modality Instance-Aware Feature Learning for 3D Multi-Object Tracking