Abstract:Indoor environment is a common scene in our everyday life, and detecting and tracking multiple targets in this environment is a key component for many applications. However, this task still remains challenging due to limited space, intrinsic target appearance variation, e. g. full or partial occlusion, large pose deformation, and scale change. In the proposed approach, we give a novel framework for detection and tracking in indoor environments, and extend it to robot navigation. One of the key components of our approach is a virtual top view created from an RGB-D camera, which is named ground plane projection (GPP). The key advantage of using GPP is the fact that the intrinsic target appearance variation and extrinsic noise is far less likely to appear in GPP than in a regular side-view image. Moreover, it is a very simple task to determine free space in GPP without any appearance learning even from a moving camera. Hence GPP is very different from the top-view image obtained from a ceiling mounted camera. We perform both object detection and tracking in GPP. Two kinds of GPP images are utilized: gray GPP, which represents the maximal height of 3D points projecting to each pixel, and binary GPP, which is obtained by thresholding the gray GPP. For detection, a simple connected component labeling is used to detect footprints of targets in binary GPP. For tracking, a novel Pixel Level Association (PLA) strategy is proposed to link the same target in consecutive frames in gray GPP. It utilizes optical flow in gray GPP, which to our best knowledge has never been done before. Then we "back project" the detected and tracked objects in GPP to original, sideview (RGB) images. Hence we are able to detect and track objects in the side-view (RGB) images. Our system is able to robustly detect and track multiple moving targets in real time. The detection process does not rely on any target model, which means we do not need any training process. Moreover, tracking does not require any manual initialization, since all entering objects are robustly detected. We also extend the novel framework to robot navigation by tracking. As our experimental results demonstrate, our approach can achieve near prefect detection and tracking results. The performance gain in comparison to state-of-the-art trackers is most significant in the presence of occlusion and background clutter.

Fusing Multimodal Video Data for Detecting Moving Objects/Targets in Challenging Indoor and Outdoor Scenes

Small Low-Contrast Target Detection: Data-Driven Spatiotemporal Feature Fusion and Implementation

Multi-spectral Image Fusion for Moving Object Detection

A Novel Framework to Localize Moving Targets in Video Surveillance Systems Via Spectral Clustering.

Detection, tracking and recognition of multi-targets using fusion data from different sensors

Multi-object Tracking Based on Thermal-Visible Video Sequence Fusion

Information Fusion of Passive Sensors for Detection of Moving Targets in Dynamic Environments.

Cognitive Fusion of Thermal and Visible Imagery for Effective Detection and Tracking of Pedestrians in Videos

FuseMODNet: Real-Time Camera and LiDAR based Moving Object Detection for robust low-light Autonomous Driving

Robust Target Detection of Intelligent Integrated Optical Camera and mmWave Radar System

Approach for moving object detection using visible spectrum and thermal infrared imaging

Deep Learning and Hybrid Approaches for Dynamic Scene Analysis, Object Detection and Motion Tracking

Object Detection in Multispectral Remote Sensing Images Based on Cross-Modal Cross-Attention

Moving target detection based on temporal-spatial information fusion for infrared image sequences

HeteroFusion: Dense Scene Reconstruction Integrating Multi-Sensors.

Spatial Pyramid Context-Aware Moving Object Detection and Tracking for Full Motion Video and Wide Aerial Motion Imagery

Multi-Sensor Data Fusion for Real-Time Multi-Object Tracking

DFECF-DET: All-Weather Detector Based on Differential Feature Enhancement and Cross-Modal Fusion With Visible and Infrared Sensors

A Novel Semantic Information Perception Architecture for Extreme Targets Detection in Complex Traffic Scenarios

Multi-object Tracking Via MHT with Multiple Information Fusion in Surveillance Video

Online Multiple Targets Detection and Tracking from Mobile Robot in Cluttered Indoor Environments with Depth Camera