Abstract:In the field of Multi-Object Tracking (MOT), the incorporation of appearance cues into tracking-by-detection heuristic trackers using re-identification (ReID) features has posed limitations on its advancement. The existing ReID paradigm involves the extraction of coarse-grained object-level feature vectors from cropped objects at a fixed input size using a ReID model, and similarity computation through a simple normalized inner product. However, MOT requires fine-grained features from different object regions and more accurate similarity measurements to identify individuals, especially in the presence of occlusion. To address these limitations, we propose a novel feature paradigm. In this paradigm, we extract the feature map from the entire frame image to preserve object sizes and represent objects using a set of fine-grained features from different object regions. These features are sampled from adaptive patches within the object bounding box on the feature map to effectively capture local appearance cues. We introduce Mutual Ratio Similarity (MRS) to accurately measure the similarity of the most discriminative region between two objects based on the sampled patches, which proves effective in handling occlusion. Moreover, we propose absolute Intersection over Union (AIoU) to consider object sizes in feature cost computation. We integrate our paradigm with advanced motion techniques to develop a heuristic Motion-Feature joint multi-object tracker, MoFe. Within it, we reformulate the track state transition of tracklets to better model their life cycle, and firstly introduce a runtime recorder after MoFe to refine trajectories. Extensive experiments on five benchmarks, i.e., GMOT-40, BDD100k, DanceTrack, MOT17, and MOT20, demonstrate that MoFe achieves state-of-the-art performance in robustness and generalizability without any fine-tuning, and even surpasses the performance of fine-tuned ReID features.

Multi-Object Tracking with Partial-Level Features and Adaptive Threshold Mechanism

MAT: Motion-Aware Multi-Object Tracking

Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking

Multi-object tracking via deep feature fusion and association analysis

MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Tracking

Multiple Objects Tracking With Improved Sparse Representation and Rank Based Dynamic Estimation.

Online multiple object tracking based on fusing global and partial features

Multi-object tracking algorithm based on interactive attention network and adaptive trajectory reconnection

Multiple Object Tracking Based on Multi‐task Learning with Strip Attention

Multi-object Tracking Combines Motion and Visual Information

Multi-object Tracking Combines Motion and Visual Information.

Multimodal Multiobject Tracking by Fusing Deep Appearance Features and Motion Information

Occlusion-Aware Detection and Re-ID Calibrated Network for Multi-Object Tracking

Motion Estimation and Multi-Stage Association for Tracking-by-detection

Single-Shot and Multi-Shot Feature Learning for Multi-Object Tracking

Addressing Challenges of Incorporating Appearance Cues Into Heuristic Multi-Object Tracker via a Novel Feature Paradigm

ByteTrack: Multi-Object Tracking by Associating Every Detection Box

Multi-Object Tracking: Decoupling Features to Solve the Contradictory Dilemma of Feature Requirements.

Real-time Multi-Object Tracking Based on Bi-directional Matching

Multi-object Tracking by Joint Detection and Identification Learning.

Spatial-Attention Location-Aware Multi-Object Tracking