Abstract:We propose a 3D multi-object tracking (MOT) solution using only 2D detections from monocular cameras, which automatically initiates/terminates tracks as well as resolves track appearance-reappearance and occlusions. Moreover, this approach does not require detector retraining when cameras are reconfigured but only the camera matrices of reconfigured cameras need to be updated. Our approach is based on a Bayesian multi-object formulation that integrates track initiation/termination, re-identification, occlusion handling, and data association into a single Bayes filtering recursion. However, the exact filter that utilizes all these functionalities is numerically intractable due to the exponentially growing number of terms in the (multi-object) filtering density, while existing approximations trade-off some of these functionalities for speed. To this end, we develop a more efficient approximation suitable for online MOT by incorporating object features and kinematics into the measurement model, which improves data association and subsequently reduces the number of terms. Specifically, we exploit the 2D detections and extracted features from multiple cameras to provide a better approximation of the multi-object filtering density to realize the track initiation/termination and re-identification functionalities. Further, incorporating a tractable geometric occlusion model based on 2D projections of 3D objects on the camera planes realizes the occlusion handling functionality of the filter. Evaluation of the proposed solution on challenging datasets demonstrates significant improvements and robustness when camera configurations change on-the-fly, compared to existing multi-view MOT solutions. The source code is publicly available at <a class="link-external link-https" href="https://github.com/linh-gist/mv-glmb-ab" rel="external noopener nofollow">this https URL</a>.

An improved deep learning architecture for multi-object tracking systems

Online Multi-Object Tracking Based on Feature Representation and Bayesian Filtering Within a Deep Learning Architecture

Joint Multi-Object Detection and Tracking with Camera-LiDAR Fusion for Autonomous Driving

InterTrack: Interaction Transformer for 3D Multi-Object Tracking

A New Architecture for Neural Enhanced Multiobject Tracking

Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Driving

An Online and Flexible Multi-object Tracking Framework Using Long Short-Term Memory

Deep Learning-Based Robust Multi-Object Tracking via Fusion of mmWave Radar and Camera Sensors

Multi-object tracking with deep learning ensemble for unmanned aerial system applications

3D LiDAR Multi-Object Tracking with Short-Term and Long-Term Multi-Level Associations

Multi-object tracking via deep feature fusion and association analysis

Joint Detection and Association for End-to-End Multi-object Tracking

Analysis Based on Recent Deep Learning Approaches Applied in Real-Time Multi-Object Tracking: A Review

Track Initialization and Re-Identification for~3D Multi-View Multi-Object Tracking

A Multi-Object Tracker Using Dynamic Bayesian Networks and a Residual Neural Network Based Similarity Estimator.

Data Association for Multi-Object Tracking via Deep Neural Networks

A Two-Stage Data Association Approach for 3D Multi-Object Tracking

Multi-Object Tracking Using Online Metric Learning with Long Short-Term Memory.

Multi-3D-Object Tracking by Fusing RGB and 3D-LiDAR Data