Abstract:Multiple object tracking (MOT) is a significant task in achieving autonomous driving. Traditional works attempt to complete this task, either based on point clouds (PC) collected by LiDAR, or based on images captured from cameras. However, relying on one single sensor is not robust enough, because it might fail during the tracking process. On the other hand, feature fusion from multiple modalities contributes to the improvement of accuracy. As a result, new techniques based on different sensors integrating features from multiple modalities are being developed. Texture information from RGB cameras and 3D structure information from Lidar have respective advantages under different circumstances. However, it's not easy to achieve effective feature fusion because of completely distinct information modalities. Previous fusion methods usually fuse the top-level features after the backbones extract the features from different modalities. In this paper, we first introduce PointNet++ to obtain multi-scale deep representations of point cloud to make it adaptive to our proposed Interactive Feature Fusion between multi-scale features of images and point clouds. Specifically, through multi-scale interactive query and fusion between pixel-level and point-level features, our method, can obtain more distinguishing features to improve the performance of multiple object tracking. Besides, we explore the effectiveness of pre-training on each single modality and fine-tuning on the fusion-based model. The experimental results demonstrate that our method can achieve good performance on the KITTI benchmark and outperform other approaches without using multi-scale feature fusion. Moreover, the ablation studies indicates the effectiveness of multi-scale feature fusion and pre-training on single modality.

Multi-scale feature fusion based on swarm intelligence collaborative learning for full-stage anti-interference object tracking

Object Tracking Model Based on Fusing Multiple Weighted Distribution Field Features

Large Margin Object Tracking with Circulant Feature Maps

Tracking an Object over 200 FPS with the Fusion of Prior Probability and Kalman Filter

Online Multi-Object Tracking from A Bird's-Eye View by Fusion of Millimeter-Wave Radar and Vision

Video object matching across multiple non-overlapping camera views based on multi-feature fusion and incremental learning.

Single-Object Tracking Algorithm Based on Two-Step Spatiotemporal Deep Feature Fusion in a Complex Surveillance Scenario

Interactive Multi-scale Fusion of 2D and 3D Features for Multi-object Tracking

A Scale Adaptive Kernel Correlation Filter Tracker With Feature Integration

Multi-object tracking via deep feature fusion and association analysis

SiamMFF: UAV Object Tracking Algorithm Based on Multi-Scale Feature Fusion

Multi-object Tracking Via MHT with Multiple Information Fusion in Surveillance Video

Real-time tracking based on deep feature fusion

Multiple object tracking with appearance feature prediction and similarity fusion

Multi-camera tracking optimization design in video surveillance system

SCSTCF: Spatial-Channel Selection and Temporal Regularized Correlation Filters for visual tracking

Anti-occlusion multi-object surveillance based on improved deep learning approach and multi-feature enhancement for unmanned smart grid safety

Robust Visual Object Tracking Based on Feature Channel Weighting and Game Theory

A Multi-Level Eigenvalue Fusion Algorithm for 3D Multi-Object Tracking

Adaptive feature fusion for visual object tracking

Long-Term Tracking Algorithm with the Combination of Multi-feature Fusion and YOLO.