AM-SORT: Adaptable Motion Predictor with Historical Trajectory Embedding for Multi-Object Tracking

Vitaliy Kim,Gunho Jung,Seong-Whan Lee
2024-01-25
Abstract:Many multi-object tracking (MOT) approaches, which employ the Kalman Filter as a motion predictor, assume constant velocity and Gaussian-distributed filtering noises. These assumptions render the Kalman Filter-based trackers effective in linear motion scenarios. However, these linear assumptions serve as a key limitation when estimating future object locations within scenarios involving non-linear motion and occlusions. To address this issue, we propose a motion-based MOT approach with an adaptable motion predictor, called AM-SORT, which adapts to estimate non-linear uncertainties. AM-SORT is a novel extension of the SORT-series trackers that supersedes the Kalman Filter with the transformer architecture as a motion predictor. We introduce a historical trajectory embedding that empowers the transformer to extract spatio-temporal features from a sequence of bounding boxes. AM-SORT achieves competitive performance compared to state-of-the-art trackers on DanceTrack, with 56.3 IDF1 and 55.6 HOTA. We conduct extensive experiments to demonstrate the effectiveness of our method in predicting non-linear movement under occlusions.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
This paper attempts to solve the limitations caused by the assumptions of linear motion and Gaussian - distributed noise in the traditional Kalman filter when multi - object tracking (MOT) involves nonlinear motion and occlusion. Specifically: 1. **Limitations of the Linear Assumption**: - The traditional Kalman filter assumes that the speed and direction of an object remain constant in a short time, and the noise in the prediction and filtering stages follows a Gaussian distribution. These assumptions make the Kalman filter very effective in handling linear motion scenarios, but it performs poorly when dealing with nonlinear motion and occlusion. - When the motion pattern of an object changes suddenly (such as a sudden change in speed or direction) or occlusion occurs, the Kalman filter cannot accurately predict the position of the object due to its linear assumption, resulting in trajectory deviation and identity mis - matching. 2. **Deficiencies of Existing Methods**: - Although the Extended Kalman Filter (EKF) and the Unscented Kalman Filter (UKF) can handle nonlinear systems to a certain extent, they still rely on linear approximation and the assumption of Gaussian - distributed noise. - The particle filter can handle nonlinear and non - Gaussian noise, but it consumes a large amount of computational resources. - Although improved methods such as OC - SORT have improved the robustness to occlusion, they still face challenges when dealing with nonlinear motion. To solve these problems, the authors propose AM - SORT (Adaptable Motion Predictor with Historical Trajectory Embedding for Multi - Object Tracking), an adaptive motion predictor based on historical trajectory embedding. The main contributions of this method include: - **Introducing an Adaptive Motion Predictor**: Use a Transformer architecture instead of the Kalman filter to capture complex time - series dependencies, so as to better handle nonlinear motion. - **Historical Trajectory Embedding**: By encoding the spatio - temporal information in the historical trajectory, enhance the model's ability to represent long - sequence trajectories and improve the robustness to nonlinear motion and occlusion. - **Relying Only on Motion Information**: Unlike traditional appearance - based methods, AM - SORT uses only bounding boxes as input features, reducing the computational complexity while maintaining high tracking accuracy. Experimental results show that AM - SORT outperforms existing motion - based multi - object tracking methods on datasets such as DanceTrack and has significant advantages in handling nonlinear motion and occlusion.