Abstract:Deep learning (DL) frameworks are widely used in various applications due to their superiority over conventional handcrafted feature-based algorithms. However, applying DL to time-range feature map-based people flow detection (PFD) with frequency-modulated continuous wave (FMCW) radar still faces several challenges: 1) simultaneously achieving people counting and motion direction recognition requires a unified framework; 2) existing mainstream network backbones designed for semantic information-rich optical images or natural language suffer performance loss in time-range feature maps with weak semantic information; and 3) the construction of labeled datasets in PFD scenes is usually costly, and limited data lead to performance loss due to overfitting. Therefore, this article proposes novel solutions from various aspects to efficiently apply DL to time-range feature map-based PFD. First, new preprocessing pipelines with Doppler spectrum analysis-based feature map truncation (DSAFMT) are proposed for the first time to simultaneously achieve people counting and direction recognition using a single radar in the radar PFD field. Second, a novel lightweight multiscale feature space fusion-based convolutional neural network backbone (MFSNet) is designed to efficiently extract multichannel differentiated representative features from time-range feature maps. Finally, a multiencoder-classifier cotraining architecture based on embedded features with two data synthesis methods and a newly designed loss function is proposed to improve the generalization ability of the algorithm. Using the test dataset collected from real scenes, the performance comparison results show that the proposed PFD algorithm outperforms the state-of-the-art algorithms and ablation studies demonstrate the effectiveness of each component of the proposed algorithm in PFD.

Multisensor Data Fusion for Life Detection in Cluttered Environments

Infrared and Radar Fusion Detection Method Based on Heterogeneous Data Preprocessing

Fusing LiDAR and Radar with Pillars Attention for 3D Object Detection

People Flow Detection Algorithm Based on a Multiencoder-Classifier Cotraining Architecture for FMCW Radar.

Life detection strategy based on infrared vision and ultra-wideband radar data fusion

Multi-parameters fusion algorithm for detecting anomalous water quality

Detection, tracking and recognition of multi-targets using fusion data from different sensors

Adaptive Multi-Pedestrian Tracking by Multi-Sensor: Track-to-Track Fusion Using Monocular 3D Detection and MMW Radar

DFECF-DET: All-Weather Detector Based on Differential Feature Enhancement and Cross-Modal Fusion With Visible and Infrared Sensors

Multi-sensor Fusion Detection Method for Vehicle Target Based on Kalman Filter and Data Association Filter

Multi-Modal and Multi-Scale Fusion 3D Object Detection of 4D Radar and LiDAR for Autonomous Driving

Airborne Small Target Detection Method Based on Multimodal and Adaptive Feature Fusion

PFEMEF-DET: All-Weather Surveillance Detector Based on Progressive Feature Enhancement and Mutual Enhancement Fusion With Visible and Infrared Sensors

Study on Airborne Multispectral Imaging Fusion Detection Technology

Unifying obstacle detection, recognition, and fusion based on millimeter wave radar and RGB-depth sensors for the visually impaired

Multi-Sensor Multi-Frame Detection Based on Posterior Probability Density Fusion.

Information Fusion of Passive Sensors for Detection of Moving Targets in Dynamic Environments.

Multilevel Structure Extraction-Based Multi-Sensor Data Fusion

Fusing Multimodal Video Data for Detecting Moving Objects/Targets in Challenging Indoor and Outdoor Scenes

Multi-sensor Data Fusion Algorithm Based on Fuzzy Adaptive Kalman Filter

A Deep Learning Method Based on the Multi-stage Fusion of Radar and Camera in UAV Obstacle Avoidance