Abstract:Accurate and reliable tracking of multi-pedestrian is of great importance for autonomous driving, human-robot interaction and video surveillance. Since different scenarios have different best-performing sensors, sensor fusion perception plans are believed to have complementary modalities and be capable of handling situations which are challenging for single sensor. In this paper, we propose a novel track-to-track fusion strategy for multi-pedestrian tracking by using a millimeter-wave (MMW) radar and a monocular camera. Pedestrians are firstly tracked by each sensor according to the sensor characteristic. Specifically, the 3D monocular pedestrian detections are obtained by a convolutional neural network (CNN). The trajectory is formed by the tracking-by-detection approach, combined with Bayesian estimation. The measurement noise of the 3D monocular detection is modeled by a detection uncertainty value obtained from the same CNN, as an approach to estimate the pedestrian state more accurately. The MMW radar utilizes the track-before-detection method due to the sparseness of the radar features. Afterwards, the pedestrian trajectories are obtained by the proposed track-to-track fusion strategy, which can work adaptively under challenging weather conditions, low-illumination conditions and clutter scenarios. A group of tests are carried out to validate our pedestrian tracking strategy. Tracking trajectories and optimal sub-pattern assignment (OSPA) metric demonstrate the accuracy and robustness of the proposed multi-sensor multi-pedestrian tracking system.

A Real-Time Active Pedestrian Tracking System Inspired by the Human Visual System

A Novel Method For Real-Time Object Detection And Multiple Persons Tracking

A Novel Approach to Design the Fast Pedestrian Detection for Video Surveillance System

Online Multipedestrian Tracking Based on Fused Detections of Millimeter Wave Radar and Vision

A Robust System for Real-Time Pedestrian Detection and Tracking

Fast Pedestrian Detection And Tracking Based On Vibe Combined Hog-Svm Scheme

Adaptive Multi-Pedestrian Tracking by Multi-Sensor: Track-to-Track Fusion Using Monocular 3D Detection and MMW Radar

Representation and Analysis of Pedestrian Crossing States Based on Video Tracking

A Real-Time System for Pedestrian Detection Based on GMM and HOG

The study of the detection and tracking of moving pedestrian using monocular-vision

Research on Vision-based Pedestrian Detection and Tracking Algorithm

A Real-Time Lidar And Vision Based Pedestrian Detection System For Unmanned Ground Vehicles

Tracking Pedestrian with Incrementally Learned Representation and Classification Model.

Pedestrian Detection and Tracking Based on Fast HOG and Time Varying System Particle Filter

Fast Pedestrian Detection and Dynamic Tracking for Intelligent Vehicles Within V2V Cooperative Environment

Study on Pedestrian Detection and Tracking with Monocular Vision

Vision Based Multi-pedestrian Tracking Using Adaptive Detection and Clustering.

Effective multiple pedestrian tracking system in video surveillance with monocular stationary camera

Combined Thermal Infrared and Visible Cameras for Pedestrian Detection and Tracking

An Improved Particle Filter Approach for Real-time Pedestrian Tracking in Surveillance Video

A Real-Time Pedestrian Counting System Based on RGB-D