Abstract:3D Multi-object tracking (MOT) ensures consistency during continuous dynamic detection, conducive to subsequent motion planning and navigation tasks in autonomous driving. However, camera-based methods suffer in the case of occlusions and it can be challenging to track the irregular motion of objects for LiDAR-based methods accurately. Some fusion methods work well but do not consider the untrustworthy issue of appearance features under occlusion. At the same time, the false detection problem also significantly affects tracking. As such, we propose a novel camera-LiDAR fusion 3D MOT framework based on Combined Appearance-Motion Optimization (CAMO-MOT), which uses both camera and LiDAR data and significantly reduces tracking failures caused by occlusion and false detection. For occlusion problems, we are the first to propose an occlusion head to select the best object appearance features multiple times effectively, reducing the influence of occlusions. To decrease the impact of false detection in tracking, we design a motion cost matrix based on confidence scores which improve the positioning and object prediction accuracy in 3D space. As existing multi-object tracking methods always evaluate each category separately and do not consider the mismatch between objects of different categories, we also propose to build a multi-category cost to implement multi-object tracking in multi-category scenes. A series of validation experiments are conducted on the KITTI and nuScenes tracking benchmarks. Our proposed method achieves state-of-the-art performance with 79.99% HOTA and the lowest identity switches (IDS) value (23 for Car and 137 for Pedestrian) among all multi-modal MOT methods on the KITTI test dataset. And our method achieves state-of-the-art performance among all algorithms on the nuScenes test dataset with 75.3% AMOTA.

Online Multiple Object Tracking Using a Novel Discriminative Module for Autonomous Driving

Online Multi-Object Tracking from A Bird's-Eye View by Fusion of Millimeter-Wave Radar and Vision

Spatial-Semantic and Temporal Attention Mechanism-Based Online Multi-Object Tracking

Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking

Distractor-aware discrimination learning for online multiple object tracking

MAT: Motion-Aware Multi-Object Tracking

Online Multi-Object Tracking Using CNN-based Single Object Tracker with Spatial-Temporal Attention Mechanism

Online Multi-Object Tracking With Visual and Radar Features

Online Multiple Object Tracking with Cross-Task Synergy

A Review of Detection-Related Multiple Object Tracking in Recent Times

Occlusion Handling Based on Motion Estimation for Multi-Object Tracking

Learning to Track: Online Multi-object Tracking by Decision Making

Multi-object tracking algorithm based on interactive attention network and adaptive trajectory reconnection

Online multiple object tracking based on fusing global and partial features

MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Tracking

ST-TrackNet: A Multiple-Object Tracking Network Using Spatio-Temporal Information

Online Multi-Object Tracking Based on Feature Representation and Bayesian Filtering Within a Deep Learning Architecture

Online multi-object tracking with unsupervised re-identification learning and occlusion estimation

FACT: Feature Adaptive Continual-learning Tracker for Multiple Object Tracking

CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking With Camera-LiDAR Fusion

3D LiDAR Multi-Object Tracking with Short-Term and Long-Term Multi-Level Associations