Abstract:Accurate and reliable tracking of multi-pedestrian is of great importance for autonomous driving, human-robot interaction and video surveillance. Since different scenarios have different best-performing sensors, sensor fusion perception plans are believed to have complementary modalities and be capable of handling situations which are challenging for single sensor. In this paper, we propose a novel track-to-track fusion strategy for multi-pedestrian tracking by using a millimeter-wave (MMW) radar and a monocular camera. Pedestrians are firstly tracked by each sensor according to the sensor characteristic. Specifically, the 3D monocular pedestrian detections are obtained by a convolutional neural network (CNN). The trajectory is formed by the tracking-by-detection approach, combined with Bayesian estimation. The measurement noise of the 3D monocular detection is modeled by a detection uncertainty value obtained from the same CNN, as an approach to estimate the pedestrian state more accurately. The MMW radar utilizes the track-before-detection method due to the sparseness of the radar features. Afterwards, the pedestrian trajectories are obtained by the proposed track-to-track fusion strategy, which can work adaptively under challenging weather conditions, low-illumination conditions and clutter scenarios. A group of tests are carried out to validate our pedestrian tracking strategy. Tracking trajectories and optimal sub-pattern assignment (OSPA) metric demonstrate the accuracy and robustness of the proposed multi-sensor multi-pedestrian tracking system.

A generalized CHNN method for track-to-track association

A Generalized CHNN Method for Track-to-track Association

A track-to-track association algorithm with chaotic neural network

Multisensor Fusion Tracking Using Continuous Dynamic Bayesian Network

Track Fusion with NN Compensated in a Multi-sensor Environment

Track correlation for multisensor system based on NN and general classical assignment algorithm

Multi-Sensors Data Tracking Fusion Based on a Neural Network Filter

Application of Neural Network in Track Correlation of Distributed Multisensor Systems

A New Multi-sensor Track Association Approach Based on Intuitionistic Fuzzy Clustering.

Sparse Representation Based Distributed Multisensor Track Fusion

Adaptive track association in distributed multi-sensor track fusion system

Multiple Sensor Track Fusion Algorithm Based on LSTM Network

Generalized Distributed Track-to-track Association Algorithm for Collaborative Network Tracking

Adaptive Multi-Pedestrian Tracking by Multi-Sensor: Track-to-Track Fusion Using Monocular 3D Detection and MMW Radar

An Adaptive Track Fusion Approach with Fuzzy Computation for Multi-sensor

Asynchronous Track-to-track Association Method in Distributed Multi-Sensor Information Fusion System

Graphical Models-Based Track Association Algorithm

A parallel centralized multi-sensor general association algorithm

The Algorithm and Simulations for the Asynchronous Track Association

Multi-target Tracking Algorithm Based on Rough and Precision Association Mixing FCM in WSN

A New Adaptive Track Correlation Method for Multiple Scenarios