Abstract:Multiple object tracking (MOT) is a significant task in achieving autonomous driving. Traditional works attempt to complete this task, either based on point clouds (PC) collected by LiDAR, or based on images captured from cameras. However, relying on one single sensor is not robust enough, because it might fail during the tracking process. On the other hand, feature fusion from multiple modalities contributes to the improvement of accuracy. As a result, new techniques based on different sensors integrating features from multiple modalities are being developed. Texture information from RGB cameras and 3D structure information from Lidar have respective advantages under different circumstances. However, it's not easy to achieve effective feature fusion because of completely distinct information modalities. Previous fusion methods usually fuse the top-level features after the backbones extract the features from different modalities. In this paper, we first introduce PointNet++ to obtain multi-scale deep representations of point cloud to make it adaptive to our proposed Interactive Feature Fusion between multi-scale features of images and point clouds. Specifically, through multi-scale interactive query and fusion between pixel-level and point-level features, our method, can obtain more distinguishing features to improve the performance of multiple object tracking. Besides, we explore the effectiveness of pre-training on each single modality and fine-tuning on the fusion-based model. The experimental results demonstrate that our method can achieve good performance on the KITTI benchmark and outperform other approaches without using multi-scale feature fusion. Moreover, the ablation studies indicates the effectiveness of multi-scale feature fusion and pre-training on single modality.

End-to-End Chained Pedestrian Multi-Object Tracking Based on Multi-Feature Fusion

Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking

Pedestrian Multi-Object Tracking with Bottleneck Transformer and Enhanced Feature Fusion

A Multi-target Tracking Method Based on Feature Enhancement

Multi-object tracking via deep feature fusion and association analysis

Multiple Object Tracking with Adaptive Multi-Features Fusion and Improved Learnable Graph Matching

Multiple object tracking with appearance feature prediction and similarity fusion

Multi-Object Tracking with Partial-Level Features and Adaptive Threshold Mechanism

Interactive Multi-scale Fusion of 2D and 3D Features for Multi-object Tracking

FAMNet: Joint Learning of Feature, Affinity and Multi-dimensional Assignment for Online Multiple Object Tracking

Online multiple object tracking based on fusing global and partial features

Mask Guided Spatial-Temporal Fusion Network for Multiple Object Tracking.

Interactive Multi-Scale Fusion of 2D and 3D Features for Multi-Object Vehicle Tracking

Learning Fully Convolutional Network for Visual Tracking with Multi-Layer Feature Fusion

Multi–feature Fusion Tracking Algorithm Based on Peak–context Learning

Triplet Network with Multi-level Feature Fusion for Object Tracking

Single-Shot and Multi-Shot Feature Learning for Multi-Object Tracking

Tracking Beyond Detection: Learning a Global Response Map for End-to-End Multi-Object Tracking

FusionTrack: Multiple Object Tracking with Enhanced Information Utilization

Joint Detection and Association for End-to-End Multi-object Tracking

Three-Dimensional Multi-Object Tracking Based on Feature Fusion and Similarity Estimation Network