Abstract:Accurate and reliable tracking of multi-pedestrian is of great importance for autonomous driving, human-robot interaction and video surveillance. Since different scenarios have different best-performing sensors, sensor fusion perception plans are believed to have complementary modalities and be capable of handling situations which are challenging for single sensor. In this paper, we propose a novel track-to-track fusion strategy for multi-pedestrian tracking by using a millimeter-wave (MMW) radar and a monocular camera. Pedestrians are firstly tracked by each sensor according to the sensor characteristic. Specifically, the 3D monocular pedestrian detections are obtained by a convolutional neural network (CNN). The trajectory is formed by the tracking-by-detection approach, combined with Bayesian estimation. The measurement noise of the 3D monocular detection is modeled by a detection uncertainty value obtained from the same CNN, as an approach to estimate the pedestrian state more accurately. The MMW radar utilizes the track-before-detection method due to the sparseness of the radar features. Afterwards, the pedestrian trajectories are obtained by the proposed track-to-track fusion strategy, which can work adaptively under challenging weather conditions, low-illumination conditions and clutter scenarios. A group of tests are carried out to validate our pedestrian tracking strategy. Tracking trajectories and optimal sub-pattern assignment (OSPA) metric demonstrate the accuracy and robustness of the proposed multi-sensor multi-pedestrian tracking system.

Multi-sensor Fusion Tracking Based on Particle Filtering and Detection Information

Detection, tracking and recognition of multi-targets using fusion data from different sensors

Multi-Sensors Data Tracking Fusion Based on a Neural Network Filter

Multi‐sensor Fusion for Multi‐target Tracking Using Measurement Division

Multisensor Fusion Tracking Using Continuous Dynamic Bayesian Network

Multisensor Multitarget Particle Filter

Multisensor Multitarget Tracking Methods Based on Particle Filter.

Multifeature Fusion Tracking in a Particle Filter Framework

Distributed Filtering Algorithm Based on Fusion of Heterogeneous Sensors' Information

A Tracking Method of Multi-sensor to Track the Multiple Targets Under the Condition of Low Detection Probability

Multi-Sensor Multi-Target Tracking Using Probability Hypothesis Density Filter

An Improved Unscented Particle Filter Approach for Multi-Sensor Fusion Target Tracking

Multi-Sensor Multi-Frame Detection Based on Posterior Probability Density Fusion.

Multisensor Information Fusion for People Tracking With a Mobile Robot: A Particle Filtering Approach

Multi-Sensor Moving Target Tracking Using Particle Filter

Adaptive Multi-Pedestrian Tracking by Multi-Sensor: Track-to-Track Fusion Using Monocular 3D Detection and MMW Radar

Particle PHD filter algorithm for multisensor multitarget tracking

Algorithm of Multi-sensor Data Fusion of PHD Filtering for Multi-target Tracking

A Robust Fusion Algorithm For Multi-Sensor Tracking

An Underwater Multisensor Multitarget Tracking Algorithm Based on Particle Filter

Multi-sensor Gaussian Mixture PHD Fusion for Multi-target Tracking