StrongSORT: Make DeepSORT Great Again

Yunhao Du,Zhicheng Zhao,Yang Song,Yanyun Zhao,Fei Su,Tao Gong,Hongying Meng

DOI: https://doi.org/10.48550/arXiv.2202.13514

2023-02-22

Abstract:Recently, Multi-Object Tracking (MOT) has attracted rising attention, and accordingly, remarkable progresses have been achieved. However, the existing methods tend to use various basic models (e.g, detector and embedding model), and different training or inference tricks, etc. As a result, the construction of a good baseline for a fair comparison is essential. In this paper, a classic tracker, i.e., DeepSORT, is first revisited, and then is significantly improved from multiple perspectives such as object detection, feature embedding, and trajectory association. The proposed tracker, named StrongSORT, contributes a strong and fair baseline for the MOT community. Moreover, two lightweight and plug-and-play algorithms are proposed to address two inherent "missing" problems of MOT: missing association and missing detection. Specifically, unlike most methods, which associate short tracklets into complete trajectories at high computation complexity, we propose an appearance-free link model (AFLink) to perform global association without appearance information, and achieve a good balance between speed and accuracy. Furthermore, we propose a Gaussian-smoothed interpolation (GSI) based on Gaussian process regression to relieve the missing detection. AFLink and GSI can be easily plugged into various trackers with a negligible extra computational cost (1.7 ms and 7.1 ms per image, respectively, on MOT17). Finally, by fusing StrongSORT with AFLink and GSI, the final tracker (StrongSORT++) achieves state-of-the-art results on multiple public benchmarks, i.e., MOT17, MOT20, DanceTrack and KITTI. Codes are available at <a class="link-external link-https" href="https://github.com/dyhBUPT/StrongSORT" rel="external noopener nofollow">this https URL</a> and <a class="link-external link-https" href="https://github.com/open-mmlab/mmtracking" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper attempts to solve several key problems in multi - object tracking (MOT):

1. **Lack of a fair benchmark**: Existing MOT methods use a variety of different base models (such as detectors and embedding models) as well as different training or inference techniques, which makes it crucial to construct a good benchmark for fair comparison.

2. **Insufficient performance of DeepSORT**: Although DeepSORT is a classic tracker, its performance is considered inferior to the latest methods. The paper believes that this is due to the relatively obsolete techniques it uses, rather than the flaws in its tracking paradigm.

3. **The "missing" problems**:
- **Missing association**: The same object may be scattered in multiple short segments, especially in online trackers, because they lack global information.
- **Missing detection**: Also known as false negatives, which refers to misidentifying an object as the background, usually caused by occlusion and low resolution.

To address these problems, the paper proposes the following solutions:

- **StrongSORT**: Significantly improve DeepSORT by introducing advanced modules (such as stronger detectors and embedding models) and some inference techniques. These improvements make it a powerful and fair benchmark in the MOT community.

- **AFLink (Appearance - Free Link Model)**: Propose a link model without appearance information, which uses spatio - temporal information to predict whether two trajectory segments belong to the same ID, thus solving the problem of missing association. AFLink achieves a good balance between speed and accuracy.

- **GSI (Gaussian - Smoothed Interpolation)**: Propose the GSI algorithm based on Gaussian process regression to alleviate the missing detection problem. GSI not only considers motion information but also improves the accuracy of the interpolation position.

Finally, by fusing StrongSORT, AFLink and GSI, the proposed final tracker (StrongSORT++) achieves state - of - the - art results in multiple public benchmark tests, including MOT17, MOT20, DanceTrack and KITTI.

In summary, the main contributions of this paper are:
- Propose StrongSORT as a powerful and fair benchmark for MOT tasks.
- Introduce two lightweight, plug - and - play algorithms, AFLink and GSI, which can significantly improve the performance of existing trackers without incurring excessive computational costs.
- Verify the effectiveness of the proposed methods through extensive experiments and achieve state - of - the - art performance in multiple benchmark tests.