Abstract:According to the real-living environment, radar-based human activity recognition (HAR) is dedicated to recognizing and classifying a sequence of activities rather than individual activities, thereby drawing more attention in practical applications of security surveillance, health care and human-computer interactions. This paper proposes a parallelism long short-term memory (LSTM) framework with the input of multi-frequency spectrograms to implement continuous HAR. Specifically, frequency-division short-time Fourier transformation (STFT) is performed on the data stream of continuous activities collected by a stepped-frequency continuous-wave (SFCW) radar, generating spectrograms of multiple frequencies which introduce different scattering properties and frequency resolutions. In the designed parallelism LSTM framework, multiple parallel LSTM sub-networks are trained separately to extract different temporal features from the spectrogram of each frequency and produce corresponding classification probabilities. At the decision level, the probabilities of activity classification from these sub-networks are fused by addition as the recognition output. To validate the proposed method, an experimental data set is collected by using an SFCW radar to monitor 11 participants who continuously perform six activities in sequence with three different transitions and random durations. The validation results demonstrate that the average accuracies of the designed parallelism unidirectional LSTM (Uni-LSTM) and bidirectional LSTM (Bi-LSTM) based on five frequency spectrograms are 85.41% and 96.15%, respectively, outperforming traditional Uni-LSTM and Bi-LSTM networks with only a single-frequency spectrogram by 5.35% and 6.33% at least. Additionally, the recognition accuracy of the parallelism LSTM network reveals an upward trend as the number of multi-frequency spectrograms (namely the number of LSTM subnetworks) increases, and tends to be stable when the number reaches 4.

Multi-frequency and multi-domain human activity recognition based on SFCW radar using deep learning

Human Activity Recognition Based on Mixed CNN With Radar Multi-Spectrogram

Human Activity Recognition Method Based on FMCW Radar Sensor with Multi-Domain Feature Attention Fusion Network

FMCW Radar Sensor Based Human Activity Recognition using Deep Learning

Sample Intercorrelation-Based Multidomain Fusion Network for Aquatic Human Activity Recognition Using Millimeter-Wave Radar.

Parallel LSTM-CNN Network with Radar Multispectrogram for Human Activity Recognition

Human Activity Recognition Based on Deep Learning and Micro-Doppler Radar Data

A Novel Multiperson Activity Recognition Algorithm Based on Point Clouds Measured by Millimeter-Wave MIMO Radar

A Novel Real-Time Multi-Person Activity Recognition System Using Millimeter-Wave SISO Radar Data

People Flow Detection Algorithm Based on a Multiencoder-Classifier Cotraining Architecture for FMCW Radar.

Human Activity Recognition Based on Multidomain Fusion Network for LFMCW Radar

Human Activity Recognition Based on Frequency-Modulated Continuous Wave and DenseNet

Radar Human Activity Recognition with an Attention-Based Deep Learning Network

Multi-domains Based Human Activity Classification in Radar

Continuous Human Activity Recognition through Parallelism LSTM with Multi-Frequency Spectrograms

A Parameter Estimation and Deep Learning Hybrid Extraction Network for Multi-Directional Human Activity Recognition Based on Mmwave Radar

Continuous Human Activity Recognition With Distributed Radar Sensor Networks and CNN–RNN Architectures

Deep Learning Techniques for Radar-Based Continuous Human Activity Recognition

Noninvasive Human Activity Recognition Using Millimeter-Wave Radar

FMCW Radar Human Action Recognition Based on Asymmetric Convolutional Residual Blocks

Sparsity-based Human Activity Recognition with PointNet using a Portable FMCW Radar