Abstract:Deep learning (DL) frameworks are widely used in various applications due to their superiority over conventional handcrafted feature-based algorithms. However, applying DL to time-range feature map-based people flow detection (PFD) with frequency-modulated continuous wave (FMCW) radar still faces several challenges: 1) simultaneously achieving people counting and motion direction recognition requires a unified framework; 2) existing mainstream network backbones designed for semantic information-rich optical images or natural language suffer performance loss in time-range feature maps with weak semantic information; and 3) the construction of labeled datasets in PFD scenes is usually costly, and limited data lead to performance loss due to overfitting. Therefore, this article proposes novel solutions from various aspects to efficiently apply DL to time-range feature map-based PFD. First, new preprocessing pipelines with Doppler spectrum analysis-based feature map truncation (DSAFMT) are proposed for the first time to simultaneously achieve people counting and direction recognition using a single radar in the radar PFD field. Second, a novel lightweight multiscale feature space fusion-based convolutional neural network backbone (MFSNet) is designed to efficiently extract multichannel differentiated representative features from time-range feature maps. Finally, a multiencoder-classifier cotraining architecture based on embedded features with two data synthesis methods and a newly designed loss function is proposed to improve the generalization ability of the algorithm. Using the test dataset collected from real scenes, the performance comparison results show that the proposed PFD algorithm outperforms the state-of-the-art algorithms and ablation studies demonstrate the effectiveness of each component of the proposed algorithm in PFD.

Multidomain Fusion Method for Human Head Movement Recognition

Sample Intercorrelation-Based Multidomain Fusion Network for Aquatic Human Activity Recognition Using Millimeter-Wave Radar.

Human Activity Recognition Based on Multidomain Fusion Network for LFMCW Radar

People Flow Detection Algorithm Based on a Multiencoder-Classifier Cotraining Architecture for FMCW Radar.

Hand Gesture Recognition Method Using FMCW Radar Based on Multidomain Fusion

A Multi-Domain Fusion Human Motion Recognition Method Based on Lightweight Network

A Robust Hand Gesture Sensing and Recognition Based on Dual-Flow Fusion with FMCW Radar.

Human Posture Recognition Based on Multi-dimensional Information Feature Fusion of Frequency Modulated Continuous Wave Radar

Human Motion Recognition With Spatial-Temporal-ConvLSTM Network Using Dynamic Range-Doppler Frames Based on Portable FMCW Radar

Human Activity Recognition Method Based on FMCW Radar Sensor with Multi-Domain Feature Attention Fusion Network

Multi-Feature Transformer-Based Learning for Continuous Human Motion Recognition with High Similarity Using mmWave FMCW Radar

Radar-Based Human Activity Recognition Using Multidomain Multilevel Fused Patch-Based Learning

A Multi-feature Fusion Temporal Neural Network for Multi-hand Gesture Recognition Using Millimeter-wave Radar Sensor

Gesture Recognition with Feature Fusion Using FMCW Radar

Continuous Human Motion Recognition with a Dynamic Range-Doppler Trajectory Method Based on FMCW Radar

Multifeature Fusion-Based Hand Gesture Sensing and Recognition System

Human Motion Recognition by Micro-Doppler Features and Concatenated CNN-LSTM Network

Dual-band polarimetric HRRP recognition via a brain-inspired multi-channel fusion feature extraction network

Human Activity Recognition Based on Data Fusion of FMCW Radar and Image

A Lightweight Network With Multifeature Fusion for mmWave Radar-Based Hand Gesture Recognition

A Lightweight Network with Multi-Feature Fusion for Mmwave Radar Based Hand Gesture Recognition