Abstract:Most existing multi-object tracking methods typically learn visual tracking features via maximizing dis-similarities of different instances and minimizing similarities of the same instance. While such a feature learning scheme achieves promising performance, learning discriminative features solely based on visual information is challenging especially in case of environmental interference such as occlusion, blur and domain variance. In this work, we argue that multi-modal language-driven features provide complementary information to classical visual features, thereby aiding in improving the robustness to such environmental interference. To this end, we propose a new multi-object tracking framework, named LG-MOT, that explicitly leverages language information at different levels of granularity (scene-and instance-level) and combines it with standard visual features to obtain discriminative representations. To develop LG-MOT, we annotate existing MOT datasets with scene-and instance-level language descriptions. We then encode both instance-and scene-level language information into high-dimensional embeddings, which are utilized to guide the visual features during training. At inference, our LG-MOT uses the standard visual features without relying on annotated language descriptions. Extensive experiments on three benchmarks, MOT17, DanceTrack and SportsMOT, reveal the merits of the proposed contributions leading to state-of-the-art performance. On the DanceTrack test set, our LG-MOT achieves an absolute gain of 2.2\% in terms of target object association (IDF1 score), compared to the baseline using only visual features. Further, our LG-MOT exhibits strong cross-domain generalizability. The dataset and code will be available at ~\url{<a class="link-external link-https" href="https://github.com/WesLee88524/LG-MOT" rel="external noopener nofollow">this https URL</a>}.

Multi-local-task Learning with Global Regularization for Object Tracking.

TLPG-Tracker: Joint Learning of Target Localization and Proposal Generation for Visual Tracking.

MLGT: multi-local guided tracker for visual object tracking

Object tracking with 3D LIDAR via multi-task sparse learning

Multi-features Guided Robust Visual Tracking.

Multi-object Tracking by Expanding Long-Tracklets

CML-MOTS: Collaborative Multi-task Learning for Multi-Object Tracking and Segmentation

Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking

GLT-T: Global-Local Transformer Voting for 3D Single Object Tracking in Point Clouds

Collaborative Deep Reinforcement Learning for Multi-object Tracking

Multi-Granularity Language-Guided Multi-Object Tracking

Learning of Global Objective for Network Flow in Multi-Object Tracking

Multi-Task Structure-Aware Context Modeling for Robust Keypoint-Based Object Tracking

Multi-Target Multi-Camera Tracking by Tracklet-to-Target Assignment

TGLC: Visual object tracking by fusion of global-local information and channel information

Multi-Agent Deep Reinforcement Learning for Multi-Object Tracker

Representation Alignment Contrastive Regularization for Multi-Object Tracking

Rlm-tracking: online multi-pedestrian tracking supported by relative location mapping

Saliency-Regularized Deep Multi-Task Learning

Effective Local and Global Search for Fast Long-term Tracking

Local to Global Tracker: A Siamese Network for Long-term Tracking