Abstract:Joint Detection and Embedding (JDE) trackers have demonstrated excellent performance in Multi-Object Tracking (MOT) tasks by incorporating the extraction of appearance features as auxiliary tasks through embedding Re-Identification task (ReID) into the detector, achieving a balance between inference speed and tracking performance. However, solving the competition between the detector and the feature extractor has always been a challenge. Meanwhile, the issue of directly embedding the ReID task into MOT has remained unresolved. The lack of high discriminability in appearance features results in their limited utility. In this paper, a new learning approach using cross-correlation to capture temporal information of objects is proposed. The feature extraction network is no longer trained solely on appearance features from each frame but learns richer motion features by utilizing feature heatmaps from consecutive frames, which addresses the challenge of inter-class feature similarity. Furthermore, our learning approach is applied to a more lightweight feature extraction network, and treat the feature matching scores as strong cues rather than auxiliary cues, with an appropriate weight calculation to reflect the compatibility between our obtained features and the MOT task. Our tracker, named TCBTrack, achieves state-of-the-art performance on multiple public benchmarks, i.e., MOT17, MOT20, and DanceTrack datasets. Specifically, on the DanceTrack test set, we achieve 56.8 HOTA, 58.1 IDF1 and 92.5 MOTA, making it the best online tracker capable of achieving real-time performance. Comparative evaluations with other trackers prove that our tracker achieves the best balance between speed, robustness and accuracy. Code is available at <a class="link-external link-https" href="https://github.com/yfzhang1214/TCBTrack" rel="external noopener nofollow">this https URL</a>.

Real-Time Multiple Object Tracking with Discriminative Features

Mean Shift Tracking Based on Ensemble Multiple Instance Learning

Temporal Correlation Meets Embedding: Towards a 2nd Generation of JDE-based Real-Time Multi-Object Tracking

Towards Real-Time Multi-Object Tracking

Real-time multiple object tracking using deep learning methods

Real-Time Online Multi-Object Tracking

Multiple Object Tracking based on Occlusion-Aware Embedding Consistency Learning

MM-Tracker: Visual Tracking with A Multi-Task Model Integrating Detection and Differentiating Feature Extraction

Object-Level Pseudo-3D Lifting for Distance-Aware Tracking

Real-Time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identification

Multiple object tracking with appearance feature prediction and similarity fusion

Real-time tracking based on deep feature fusion

Multi-object tracking via deep feature fusion and association analysis

Multi-Stream Siamese and Faster Region-Based Neural Network for Real-Time Object Tracking

Instance Segmentation Enabled Hybrid Data Association and Discriminative Hashing for Online Multi-Object Tracking

DEFT: Detection Embeddings for Tracking

Multi-object Tracking by Expanding Long-Tracklets

Distractor-aware discrimination learning for online multiple object tracking

Long-term tracking algorithm using deep features and a single shot multibox detector.

Real-time Multi-Object Tracking Based on Bi-directional Matching

Real-time Object Tracking Based on Improved Adversarial Learning