Abstract:Traffic target tracking is a core task in intelligent transportation system because it is useful for scene understanding and vehicle autonomous driving. Most state-of-the-art (SOTA) multiple object tracking (MOT) methods adopt a two-step procedure: object detection followed by data association. The object detection has made great progress with the development of deep learning. However, the data association still heavily depends on hand crafted constraints, such as appearance, shape, and motion, which need to be elaborately trained for a special object. In this study, a spatial-temporal encoder-decoder affinity network is proposed for multiple traffic targets tracking, aiming to utilize the power of deep learning to learn a robust spatial-temporal affinity feature of the detections and tracklets for data association. The proposed spatial-temporal affinity network contains a two-stage transformer encoder module to encode the features of the detections and the tracked targets at the image level and the tracklet level, aiming to capture the spatial correlation and temporal history information. Then, a spatial transformer decoder module is designed to compute the association affinity, where the results from the two-stage transformer encoder module are fed back to fully capture and encode the spatial and temporal information from the detections and the tracklets of the tracked targets. Thus, efficient affinity computation can be applied to perform data association in online tracking. To validate the effectiveness of the proposed method, three popular multiple traffic target tracking datasets, KITTI, UA-DETRAC, and VisDrone, are used for evaluation. On the KITTI dataset, the proposed method is compared with 15 SOTA methods and achieves 86.9% multiple object tracking accuracy (MOTA) and 85.71% multiple object tracking precision (MOTP). On the UA-DETRAC dataset, 12 SOTA methods are used to compare with the proposed method, and the proposed method achieves 20.82% MOTA and 35.65% MOTP, respectively. On the VisDrone dataset, the proposed method is compared with 10 SOTA trackers and achieves 40.5% MOTA and 74.1% MOTP, respectively. All those experimental results show that the proposed method is competitive to the state-of-the-art methods by obtaining superior tracking performance.

Multiple object detection and tracking from drone videos based on GM-YOLO and multi-tracker

Online Multi-Object Tracking from A Bird's-Eye View by Fusion of Millimeter-Wave Radar and Vision

Exploit the Connectivity: Multi-Object Tracking with TrackletNet

Multi-Object Tracking Meets Moving UAV

GIAOTracker: A comprehensive framework for MCMOT with global information and optimizing strategies in VisDrone 2021

Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking

Multi-Drone-Based Single Object Tracking With Agent Sharing Network

Multiple Object Tracking of Drone Videos by a Temporal-Association Network with Separated-Tasks Structure

Multiple Object Tracking with Motion and Appearance Cues

A Multi-Object Tracker Using Dynamic Bayesian Networks and a Residual Neural Network Based Similarity Estimator.

Enhancing Online UAV Multi-Object Tracking with Temporal Context and Spatial Topological Relationships

Yolo-3DMM for Simultaneous Multiple Object Detection and Tracking in Traffic Scenarios

Single-Shot and Multi-Shot Feature Learning for Multi-Object Tracking

Multi-Object Vehicle Detection and Tracking Algorithm Based on Improved YOLOv8 and ByteTrack

End-to-end multiple object tracking in high-resolution optical sensors of drones with transformer models

Object Tracking in Unmanned Aerial Vehicle Videos via Multifeature Discrimination and Instance-Aware Attention Network

Mask Guided Spatial-Temporal Fusion Network for Multiple Object Tracking.

Multi-object tracking in UAVs with feature fusion distribution and occlusion awareness

Robust Multi-Drone Multi-Target Tracking to Resolve Target Occlusion: A Benchmark

Multiple Traffic Target Tracking with Spatial-Temporal Affinity Network

Multi-object tracking via deep feature fusion and association analysis