Abstract:Joint Detection and Embedding (JDE) trackers have demonstrated excellent performance in Multi-Object Tracking (MOT) tasks by incorporating the extraction of appearance features as auxiliary tasks through embedding Re-Identification task (ReID) into the detector, achieving a balance between inference speed and tracking performance. However, solving the competition between the detector and the feature extractor has always been a challenge. Meanwhile, the issue of directly embedding the ReID task into MOT has remained unresolved. The lack of high discriminability in appearance features results in their limited utility. In this paper, a new learning approach using cross-correlation to capture temporal information of objects is proposed. The feature extraction network is no longer trained solely on appearance features from each frame but learns richer motion features by utilizing feature heatmaps from consecutive frames, which addresses the challenge of inter-class feature similarity. Furthermore, our learning approach is applied to a more lightweight feature extraction network, and treat the feature matching scores as strong cues rather than auxiliary cues, with an appropriate weight calculation to reflect the compatibility between our obtained features and the MOT task. Our tracker, named TCBTrack, achieves state-of-the-art performance on multiple public benchmarks, i.e., MOT17, MOT20, and DanceTrack datasets. Specifically, on the DanceTrack test set, we achieve 56.8 HOTA, 58.1 IDF1 and 92.5 MOTA, making it the best online tracker capable of achieving real-time performance. Comparative evaluations with other trackers prove that our tracker achieves the best balance between speed, robustness and accuracy. Code is available at <a class="link-external link-https" href="https://github.com/yfzhang1214/TCBTrack" rel="external noopener nofollow">this https URL</a>.

Kernel Joint Visual Tracking and Recognition Based on Structured Sparse Representation.

Robust Visual Tracking Via CAMShift and Structural Local Sparse Appearance Model

Online Learning and Joint Optimization of Combined Spatial-Temporal Models for Robust Visual Tracking.

Robust Visual Tracking Using Sparse Discriminative Graph Embedding.

Visual tracking via robust multi-task multi-feature joint sparse representation

Robust Visual Tracking Using Multi-Frame Multi-Feature Joint Modeling.

Kernel Based Visual Tracking with Variant Spatial Resolution Model

Visual Tracking Via Joint Discriminative Appearance Learning

Tracking via Robust Multi-task Multi-view Joint Sparse Representation

Visual Tracking Via Efficient Kernel Discriminant Subspace Learning

Robust Visual Cooperative Tracking Using Constrained Adaptive Sparse Representations and Sparse Classifier Grids

Structured sparse representation appearance model for robust visual tracking

Robust Fragments-Based Tracking with Multi-Feature Joint Kernel Sparse Representation

Robust Visual Tracking Using an Effective Appearance Model Based on Sparse Coding.

Generalized Kernel-Based Visual Tracking.

Collaborative object tracking model with local sparse representation

Joint Spatio-Temporal Similarity and Discrimination Learning for Visual Tracking

Multi-cue Visual Tracking Based on Sparse Representation

Temporal Correlation Meets Embedding: Towards a 2nd Generation of JDE-based Real-Time Multi-Object Tracking

Weighted Joint Sparse Representation Based Visual Tracking

Visual Object Tracking Using Structured Sparse PCA-Based Appearance Representation and Online Learning