Abstract:Modern multiple object tracking (MOT) systems usually follow the tracking-by-detection paradigm. It has 1) a detection model for target localization and 2) an appearance embedding model for data association. Having the two models separately executed might lead to efficiency problems, as the running time is simply a sum of the two steps without investigating potential structures that can be shared between them. Existing research efforts on real-time MOT usually focus on the association step, so they are essentially real-time association methods but not real-time MOT system. In this paper, we propose an MOT system that allows target detection and appearance embedding to be learned in a shared model. Specifically, we incorporate the appearance embedding model into a single-shot detector, such that the model can simultaneously output detections and the corresponding embeddings. We further propose a simple and fast association method that works in conjunction with the joint model. In both components the computation cost is significantly reduced compared with former MOT systems, resulting in a neat and fast baseline for future follow-ups on real-time MOT algorithm design. To our knowledge, this work reports the first (near) real-time MOT system, with a running speed of 22 to 40 FPS depending on the input resolution. Meanwhile, its tracking accuracy is comparable to the state-of-the-art trackers embodying separate detection and embedding (SDE) learning (64.4%$$64.4\%$$ MOTA v.s. 66.1%$$66.1\%$$ MOTA on MOT-16 challenge). Code and models are available at https://github.com/Zhongdao/Towards-Realtime-MOT.

Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking

Split and Connect: A Universal Tracklet Booster for Multi-Object Tracking

MAT: Motion-Aware Multi-Object Tracking

MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectors

Multi-Object Tracking with Partial-Level Features and Adaptive Threshold Mechanism

Multi-Object Tracking Via Multi-Attention

MOTR: End-to-End Multiple-Object Tracking with Transformer

Spatial-Attention Location-Aware Multi-Object Tracking

Joint Detection and Association for End-to-End Multi-object Tracking

Spatial-Semantic and Temporal Attention Mechanism-Based Online Multi-Object Tracking

Multiple Object Tracking Based on Multi‐task Learning with Strip Attention

Towards Real-Time Multi-Object Tracking

Online Multi-Object Tracking with Dual Matching Attention Networks

An Improved Unified Online Multi-Object Tracking Algorithm Combined with Attention Mechanism

MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Tracking

Tracking Beyond Detection: Learning a Global Response Map for End-to-End Multi-Object Tracking

End-to-End Chained Pedestrian Multi-Object Tracking Based on Multi-Feature Fusion

ByteTrack: Multi-Object Tracking by Associating Every Detection Box

Multi-object tracking algorithm based on interactive attention network and adaptive trajectory reconnection

Real-time Multi-Object Tracking Based on Bi-directional Matching

Occlusion-Aware Detection and Re-ID Calibrated Network for Multi-Object Tracking