Abstract:In the field of Multi-Object Tracking (MOT), the incorporation of appearance cues into tracking-by-detection heuristic trackers using re-identification (ReID) features has posed limitations on its advancement. The existing ReID paradigm involves the extraction of coarse-grained object-level feature vectors from cropped objects at a fixed input size using a ReID model, and similarity computation through a simple normalized inner product. However, MOT requires fine-grained features from different object regions and more accurate similarity measurements to identify individuals, especially in the presence of occlusion. To address these limitations, we propose a novel feature paradigm. In this paradigm, we extract the feature map from the entire frame image to preserve object sizes and represent objects using a set of fine-grained features from different object regions. These features are sampled from adaptive patches within the object bounding box on the feature map to effectively capture local appearance cues. We introduce Mutual Ratio Similarity (MRS) to accurately measure the similarity of the most discriminative region between two objects based on the sampled patches, which proves effective in handling occlusion. Moreover, we propose absolute Intersection over Union (AIoU) to consider object sizes in feature cost computation. We integrate our paradigm with advanced motion techniques to develop a heuristic Motion-Feature joint multi-object tracker, MoFe. Within it, we reformulate the track state transition of tracklets to better model their life cycle, and firstly introduce a runtime recorder after MoFe to refine trajectories. Extensive experiments on five benchmarks, i.e., GMOT-40, BDD100k, DanceTrack, MOT17, and MOT20, demonstrate that MoFe achieves state-of-the-art performance in robustness and generalizability without any fine-tuning, and even surpasses the performance of fine-tuned ReID features.

Feature Disentanglement Network: Multi-Object Tracking Needs More Differentiated Features

Exploit the Connectivity: Multi-Object Tracking with TrackletNet

Exploit the Connectivity

Split and Connect: A Universal Tracklet Booster for Multi-Object Tracking

Decoupling Features Via Global and Local Contexts in Multi-Object Tracking

Rethinking the Competition Between Detection and ReID in Multiobject Tracking

MAT: Motion-Aware Multi-Object Tracking

RelationTrack: Relation-aware Multiple Object Tracking with Decoupled Representation

Multi-object Tracking with a Hierarchical Single-branch Network

Multi-object tracking via deep feature fusion and association analysis

InterTrack: Interaction Transformer for 3D Multi-Object Tracking

Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking

Addressing Challenges of Incorporating Appearance Cues Into Heuristic Multi-Object Tracker via a Novel Feature Paradigm

Focus On Details: Online Multi-Object Tracking with Diverse Fine-Grained Representation

Fusion detection and ReID embedding with hybrid attention for multi-object tracking

MM-Tracker: Visual Tracking with A Multi-Task Model Integrating Detection and Differentiating Feature Extraction

Single-Shot and Multi-Shot Feature Learning for Multi-Object Tracking

SCGTracker: object feature embedding enhancement based on graph attention networks for multi-object tracking

Online Multiple Object Tracking with Enhanced Re‐identification

Online Multi-Object Tracking Based on Feature Representation and Bayesian Filtering Within a Deep Learning Architecture