Abstract:Multiple-object tracking (MOT) is a crucial component in autonomous driving systems. However, inaccurate object detection is always the bottleneck for MOT. Most detectors are not designed to take the temporal information across consecutive frames into consideration. To take advantage of such information, we design a novel data representation, the spatio-temporal (ST) map, which collects a batch of detection results spatio-temporally, and we train a novel network, ST-TrackNet, to assign predicted track IDs to each positive detection across a sequence. With our ST map detection fed into the tracker, the correlation of objects between adjacent frames becomes prominent, which improves the performance of the tracker in the data association step. Moreover, the long-term trajectory in a sequence also helps to refine the detection results. We train and evaluate our network on the KITTI dataset, a CARLA simulation dataset, and a dataset recorded in a factory environment. Our approach generally achieves superior performance over the state-of-the-art. Note to Practitioners—We investigate the MOT problem in this paper. A spatio-temporal pipeline is proposed to provide a solution to this problem. Object detection results produced by off-the-shelf object detectors are used to form the proposed ST maps. In low signal-to-noise ratio (SNR) situations, our proposed framework can achieve more accurate and robust tracking results with more false-positives. Due to the simplicity and modular design of our framework, it can be applied directly after the detection stage to achieve the online tracking task. The proposed method is evaluated on several datasets, and the experimental results demonstrate its effectiveness. Our method can also be used for other autonomous driving applications, such as path planning and trajectory prediction.

The Application Of Neural Network In Multiple Object Tracking

Exploit the Connectivity: Multi-Object Tracking with TrackletNet

Convolutional Neural Network-based Simple Online Multiple Object Tracking

Deep Learning-based Visual Multiple Object Tracking:A Review

Analysis Based on Recent Deep Learning Approaches Applied in Real-Time Multi-Object Tracking: A Review

Multiple Object Tracking in Deep Learning Approaches: A Survey

A Review of Detection-Related Multiple Object Tracking in Recent Times

Enhanced Multi-Object Tracking: Inferring Motion States of Tracked Objects

Multiple Object Tracking Using a Dual‐attention Network for Autonomous Driving

Multi-object Tracking by Joint Detection and Identification Learning.

Graph Attention Networks and Track Management for Multiple Object Tracking

Online Multi-Object Tracking with Convolutional Neural Networks.

Deep learning and multi-modal fusion for real-time multi-object tracking: Algorithms, challenges, datasets, and comparative study

Multimodal Multiobject Tracking by Fusing Deep Appearance Features and Motion Information

Deep Human-Interaction and Association by Graph-Based Learning for Multiple Object Tracking in the Wild

Multiple Hypothesis Tracking Based on Discriminative Appearance Features of Convolutional Neural Network

Spatio-Temporal Point Process for Multiple Object Tracking

ST-TrackNet: A Multiple-Object Tracking Network Using Spatio-Temporal Information

A New Architecture for Neural Enhanced Multiobject Tracking