Abstract:Multiple Object Tracking (MOT) in thermal imaging presents unique challenges due to the lack of visual features and the complexity of motion patterns. This paper introduces an innovative approach to improve MOT in the thermal domain by developing a novel box association method that utilizes both thermal object identity and motion similarity. Our method merges thermal feature sparsity and dynamic object tracking, enabling more accurate and robust MOT performance. Additionally, we present a new dataset comprised of a large-scale collection of thermal and RGB images captured in diverse urban environments, serving as both a benchmark for our method and a new resource for thermal imaging. We conduct extensive experiments to demonstrate the superiority of our approach over existing methods, showing significant improvements in tracking accuracy and robustness under various conditions. Our findings suggest that incorporating thermal identity with motion data enhances MOT performance. The newly collected dataset and source code is available at <a class="link-external link-https" href="https://github.com/wassimea/thermalMOT" rel="external noopener nofollow">this https URL</a>

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the unique challenges faced in multi - object tracking (MOT) in thermal imaging. Specifically, due to the lack of visual features and complex motion patterns in thermal imaging, the performance of traditional MOT methods in thermal imaging is often poor. To solve these problems, the author proposes a new box - association method, which combines the identity of thermal objects and motion similarity to improve the performance of MOT in thermal imaging. ### Summary of Main Problems in the Paper 1. **Unique Challenges of MOT in Thermal Imaging**: - **Lack of Visual Features**: Thermal imaging cannot provide the same rich visual features as visible - light images, making it difficult for traditional MOT methods based on visual features to be directly applied. - **Complex Motion Patterns**: The motion patterns of objects in thermal imaging are more complex, increasing the difficulty of tracking. 2. **Limitations of Existing Methods**: - Existing MOT methods mainly rely on motion association (such as using the Kalman filter to predict object positions), but this method does not fully utilize the unique characteristics of thermal imaging (such as the identity of thermal objects). 3. **Lack of Datasets**: - The lack of large - scale public thermal - imaging MOT datasets limits the development of related research. ### Proposed Solutions To address the above challenges, the author proposes the following solutions: 1. **New Box - Association Method**: - By combining the identity of thermal objects and motion similarity, a new box - association algorithm is proposed, which improves the accuracy and robustness of MOT. 2. **New Dataset**: - A large - scale dataset containing RGB and thermal imaging is constructed, covering a variety of urban environments, providing a valuable resource for MOT research. 3. **Experimental Verification**: - Extensive experiments are carried out on the new dataset, demonstrating the superiority of the proposed method under different conditions. ### Formula Presentation The formulas involved in the paper are mainly used to describe the similarity calculation in the box - association algorithm. For example, the calculation formula of the comprehensive similarity matrix \( S_{\text{comp}} \) is as follows: \[ S_{\text{comp}} = \alpha \cdot S_{\text{motion}}+(1 - \alpha) \cdot S_{\text{thermal}} \] where: - \( S_{\text{motion}} \) is the motion - based similarity matrix, - \( S_{\text{thermal}} \) is the thermal - imaging - based similarity matrix, - \( \alpha \) is a weighting factor used to balance the contributions of motion similarity and thermal similarity. By adjusting the value of \( \alpha \), the best performance balance point can be found in different trackers (such as ByteTrack and OCSORT).

Enhancing Thermal MOT: A Novel Box Association Method Leveraging Thermal Identity and Motion Similarity

Visible-Thermal Multiple Object Tracking: Large-scale Video Dataset and Progressive Fusion Approach

MAT: Motion-Aware Multi-Object Tracking

Exploit the Connectivity

Online Multi-Object Tracking from A Bird's-Eye View by Fusion of Millimeter-Wave Radar and Vision

Exploit the Connectivity: Multi-Object Tracking with TrackletNet

IA-MOT: Instance-Aware Multi-Object Tracking with Motion Consistency

Deep Efficient Data Association for Multi-Object Tracking: Augmented with SSIM-Based Ambiguity Elimination

MOT FCG++: Enhanced Representation of Spatio-temporal Motion and Appearance Features

MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Tracking

Appearance-free Tripartite Matching for Multiple Object Tracking

DeepFusionMOT: A 3D Multi-Object Tracking Framework Based on Camera-LiDAR Fusion With Deep Association

CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking With Camera-LiDAR Fusion

Learning Deep Multi-Level Similarity for Thermal Infrared Object Tracking

Track Initialization and Re-Identification for~3D Multi-View Multi-Object Tracking

Attention Modulated Multiple Object Tracking with Motion Enhancement and Dual Correlation

TOPIC: A Parallel Association Paradigm for Multi-Object Tracking under Complex Motions and Diverse Scenes

STMMOT: Advancing multi-object tracking through spatiotemporal memory networks and multi-scale attention pyramids

Motion State: A New Benchmark Multiple Object Tracking

Deep Affinity Network for Multiple Object Tracking

EagerMOT: 3D Multi-Object Tracking via Sensor Fusion