Abstract:Deep learning (DL) frameworks are widely used in various applications due to their superiority over conventional handcrafted feature-based algorithms. However, applying DL to time-range feature map-based people flow detection (PFD) with frequency-modulated continuous wave (FMCW) radar still faces several challenges: 1) simultaneously achieving people counting and motion direction recognition requires a unified framework; 2) existing mainstream network backbones designed for semantic information-rich optical images or natural language suffer performance loss in time-range feature maps with weak semantic information; and 3) the construction of labeled datasets in PFD scenes is usually costly, and limited data lead to performance loss due to overfitting. Therefore, this article proposes novel solutions from various aspects to efficiently apply DL to time-range feature map-based PFD. First, new preprocessing pipelines with Doppler spectrum analysis-based feature map truncation (DSAFMT) are proposed for the first time to simultaneously achieve people counting and direction recognition using a single radar in the radar PFD field. Second, a novel lightweight multiscale feature space fusion-based convolutional neural network backbone (MFSNet) is designed to efficiently extract multichannel differentiated representative features from time-range feature maps. Finally, a multiencoder-classifier cotraining architecture based on embedded features with two data synthesis methods and a newly designed loss function is proposed to improve the generalization ability of the algorithm. Using the test dataset collected from real scenes, the performance comparison results show that the proposed PFD algorithm outperforms the state-of-the-art algorithms and ablation studies demonstrate the effectiveness of each component of the proposed algorithm in PFD.

Radar-based Human Activity Recognition Using Two-Dimensional Feature Extraction

A Novel Multiperson Activity Recognition Algorithm Based on Point Clouds Measured by Millimeter-Wave MIMO Radar

A Novel Real-Time Multi-Person Activity Recognition System Using Millimeter-Wave SISO Radar Data

People Flow Detection Algorithm Based on a Multiencoder-Classifier Cotraining Architecture for FMCW Radar.

Point Cloud Features-Based Kernel SVM for Human-Vehicle Classification in Millimeter Wave Radar.

Human Recognition with the Optoelectronic Reservoir-Computing-based Micro-Doppler Radar Signal Processing.

Noninvasive Human Activity Recognition Using Millimeter-Wave Radar

A Parameter Estimation and Deep Learning Hybrid Extraction Network for Multi-Directional Human Activity Recognition Based on Mmwave Radar

Multi-frequency and multi-domain human activity recognition based on SFCW radar using deep learning

Human Activity Recognition Method Based on FMCW Radar Sensor with Multi-Domain Feature Attention Fusion Network

Multi-domains Based Human Activity Classification in Radar

Human Activity Recognition Based on Deep Learning and Micro-Doppler Radar Data

Human Activity Recognition Based on Mixed CNN With Radar Multi-Spectrogram

Multi-Classification Algorithm for Human Motion Recognition Based on IR-UWB Radar

Human Behavior Recognition Based on Multi-Dimensional Feature Learning of Millimeter-Wave Radar

Human Activity Recognition Based on Frequency-Modulated Continuous Wave and DenseNet

Human Multi-Activities Classification Using mmWave Radar: Feature Fusion in Time-Domain and PCANet

Parallel LSTM-CNN Network with Radar Multispectrogram for Human Activity Recognition

FMCW Radar Sensor Based Human Activity Recognition using Deep Learning

Real-time Human Activity Classification from Radar with CNN-LSTM Network *

4D radar simulator for human activity recognition