Abstract:Multiple object tracking (MOT) is a significant task in achieving autonomous driving. Traditional works attempt to complete this task, either based on point clouds (PC) collected by LiDAR, or based on images captured from cameras. However, relying on one single sensor is not robust enough, because it might fail during the tracking process. On the other hand, feature fusion from multiple modalities contributes to the improvement of accuracy. As a result, new techniques based on different sensors integrating features from multiple modalities are being developed. Texture information from RGB cameras and 3D structure information from Lidar have respective advantages under different circumstances. However, it's not easy to achieve effective feature fusion because of completely distinct information modalities. Previous fusion methods usually fuse the top-level features after the backbones extract the features from different modalities. In this paper, we first introduce PointNet++ to obtain multi-scale deep representations of point cloud to make it adaptive to our proposed Interactive Feature Fusion between multi-scale features of images and point clouds. Specifically, through multi-scale interactive query and fusion between pixel-level and point-level features, our method, can obtain more distinguishing features to improve the performance of multiple object tracking. Besides, we explore the effectiveness of pre-training on each single modality and fine-tuning on the fusion-based model. The experimental results demonstrate that our method can achieve good performance on the KITTI benchmark and outperform other approaches without using multi-scale feature fusion. Moreover, the ablation studies indicates the effectiveness of multi-scale feature fusion and pre-training on single modality.

Multi-Sensor Multi-Object Tracking with Different Fields-of-View Using the LMB Filter.

Online Multi-Object Tracking from A Bird's-Eye View by Fusion of Millimeter-Wave Radar and Vision

Distributed Fusion with Multi-Bernoulli Filter based on Generalized Covariance Intersection

Computationally Efficient Distributed Multi-sensor Fusion with Multi-Bernoulli Filter

Multi-Sensors Data Tracking Fusion Based on a Neural Network Filter

Detection, tracking and recognition of multi-targets using fusion data from different sensors

Distributed Multi-Object Tracking Under Limited Field of View Heterogeneous Sensors with Density Clustering

Multi-person Multi-Camera Tracking for Live Stream Videos Based on Improved Motion Model and Matching Cascade

MMF-Track: Multi-modal Multi-level Fusion for 3D Single Object Tracking

Object Tracking over Multiple Uncalibrated Cameras Using Visual, Spatial and Temporal Similarities.

Multi-features Guided Robust Visual Tracking.

Research on multi-target tracking method based on multi-sensor fusion

Multi-object Tracking Via MHT with Multiple Information Fusion in Surveillance Video

Multi-view tracking in wireless sensor networks

3D Multiple Object Tracking with Multi-modal Fusion of Low-cost Sensors for Autonomous Driving.

MFITrack: Multi-Frame Integration Strategy for Enhanced Motion-Centric Single Object Tracking

Interactive Multi-scale Fusion of 2D and 3D Features for Multi-object Tracking

A Multi-Level Eigenvalue Fusion Algorithm for 3D Multi-Object Tracking

Distributed Multi-Sensor Fusion of PHD Filters With Different Sensor Fields of View

Distributed GGIW-CPHD-Based Extended Target Tracking over a Sensor Network