Abstract:Deep learning (DL) frameworks are widely used in various applications due to their superiority over conventional handcrafted feature-based algorithms. However, applying DL to time-range feature map-based people flow detection (PFD) with frequency-modulated continuous wave (FMCW) radar still faces several challenges: 1) simultaneously achieving people counting and motion direction recognition requires a unified framework; 2) existing mainstream network backbones designed for semantic information-rich optical images or natural language suffer performance loss in time-range feature maps with weak semantic information; and 3) the construction of labeled datasets in PFD scenes is usually costly, and limited data lead to performance loss due to overfitting. Therefore, this article proposes novel solutions from various aspects to efficiently apply DL to time-range feature map-based PFD. First, new preprocessing pipelines with Doppler spectrum analysis-based feature map truncation (DSAFMT) are proposed for the first time to simultaneously achieve people counting and direction recognition using a single radar in the radar PFD field. Second, a novel lightweight multiscale feature space fusion-based convolutional neural network backbone (MFSNet) is designed to efficiently extract multichannel differentiated representative features from time-range feature maps. Finally, a multiencoder-classifier cotraining architecture based on embedded features with two data synthesis methods and a newly designed loss function is proposed to improve the generalization ability of the algorithm. Using the test dataset collected from real scenes, the performance comparison results show that the proposed PFD algorithm outperforms the state-of-the-art algorithms and ablation studies demonstrate the effectiveness of each component of the proposed algorithm in PFD.

Radar-Based Human Activity Recognition Using Multidomain Multilevel Fused Patch-Based Learning

Sample Intercorrelation-Based Multidomain Fusion Network for Aquatic Human Activity Recognition Using Millimeter-Wave Radar.

People Flow Detection Algorithm Based on a Multiencoder-Classifier Cotraining Architecture for FMCW Radar.

A Novel Potential Drowning Detection System Based on Millimeter-Wave Radar

A Novel Multiperson Activity Recognition Algorithm Based on Point Clouds Measured by Millimeter-Wave MIMO Radar

Radar Human Activity Recognition with an Attention-Based Deep Learning Network

Human Activity Recognition Method Based on FMCW Radar Sensor with Multi-Domain Feature Attention Fusion Network

Multi-frequency and multi-domain human activity recognition based on SFCW radar using deep learning

DAMUN: A Domain Adaptive Human Activity Recognition Network Based on Multimodal Feature Fusion

Human Activity Recognition Based on Mixed CNN With Radar Multi-Spectrogram

Human Activity Recognition Based on Deep Learning and Micro-Doppler Radar Data

Narrowband Radar Automatic Target Recognition Based on a Hierarchical Fusing Network With Multidomain Features

FMCW Radar Sensor Based Human Activity Recognition using Deep Learning

A lightweight hybrid vision transformer network for radar-based human activity recognition

A Multi-Domain Fusion Human Motion Recognition Method Based on Lightweight Network

Continuous Human Activity Recognition With Distributed Radar Sensor Networks and CNN–RNN Architectures

Dual-band polarimetric HRRP recognition via a brain-inspired multi-channel fusion feature extraction network

Indoor Activity and Vital Sign Monitoring for Moving People with Multiple Radar Data Fusion

A Multimodal Data Processing System for LiDAR-Based Human Activity Recognition

Multi-Sensor Data Fusion and CNN-LSTM Model for Human Activity Recognition System

Angle-Insensitive Human Motion and Posture Recognition Based on 4D Imaging Radar and Deep Learning Classifiers