Abstract:Joint Detection and Embedding (JDE) trackers have demonstrated excellent performance in Multi-Object Tracking (MOT) tasks by incorporating the extraction of appearance features as auxiliary tasks through embedding Re-Identification task (ReID) into the detector, achieving a balance between inference speed and tracking performance. However, solving the competition between the detector and the feature extractor has always been a challenge. Meanwhile, the issue of directly embedding the ReID task into MOT has remained unresolved. The lack of high discriminability in appearance features results in their limited utility. In this paper, a new learning approach using cross-correlation to capture temporal information of objects is proposed. The feature extraction network is no longer trained solely on appearance features from each frame but learns richer motion features by utilizing feature heatmaps from consecutive frames, which addresses the challenge of inter-class feature similarity. Furthermore, our learning approach is applied to a more lightweight feature extraction network, and treat the feature matching scores as strong cues rather than auxiliary cues, with an appropriate weight calculation to reflect the compatibility between our obtained features and the MOT task. Our tracker, named TCBTrack, achieves state-of-the-art performance on multiple public benchmarks, i.e., MOT17, MOT20, and DanceTrack datasets. Specifically, on the DanceTrack test set, we achieve 56.8 HOTA, 58.1 IDF1 and 92.5 MOTA, making it the best online tracker capable of achieving real-time performance. Comparative evaluations with other trackers prove that our tracker achieves the best balance between speed, robustness and accuracy. Code is available at <a class="link-external link-https" href="https://github.com/yfzhang1214/TCBTrack" rel="external noopener nofollow">this https URL</a>.

A Twofold Convolutional Regression Tracking Network with Temporal and Spatial Mechanism

Exploit Spatiotemporal Contextual Information for 3D Single Object Tracking Via Memory Networks

Robust Online Tracking Via Contrastive Spatio-Temporal Aware Network.

Robust Visual Object Tracking with Two-Stream Residual Convolutional Networks

Continuity-Discrimination Convolutional Neural Network for Visual Object Tracking

Dual Deep Network for Visual Tracking

Dynamic memory network with spatial-temporal feature fusion for visual tracking

Real Time Visual Tracking using Spatial-Aware Temporal Aggregation Network

Residual Learning-Based Two-Stream Network for RGB-T Object Tracking

Modeling of Multiple Spatial-Temporal Relations for Robust Visual Object Tracking

Temporal Correlation Meets Embedding: Towards a 2nd Generation of JDE-based Real-Time Multi-Object Tracking

Robust Visual Tracking Via Collaborative and Reinforced Convolutional Feature Learning

Learning Attribute-Specific Representations for Visual Tracking.

Visual Tracking with Long-Short Term Based Correlation Filter

Hierarchical Convolutional Features for Visual Tracking

Learning Regression and Verification Networks for Robust Long-term Tracking

Learning Transform-Aware Attentive Network for Object Tracking

Dual Integration of Multi‐model with Spatial‐temporal Occlusion‐awareness for Visual Object Tracking

Adaptive multi-branch correlation filters for robust visual tracking

Robust Visual Tracking Via Hierarchical Convolutional Features

Towards Real-World Visual Tracking with Temporal Contexts