Abstract:According to the real-living environment, radar-based human activity recognition (HAR) is dedicated to recognizing and classifying a sequence of activities rather than individual activities, thereby drawing more attention in practical applications of security surveillance, health care and human-computer interactions. This paper proposes a parallelism long short-term memory (LSTM) framework with the input of multi-frequency spectrograms to implement continuous HAR. Specifically, frequency-division short-time Fourier transformation (STFT) is performed on the data stream of continuous activities collected by a stepped-frequency continuous-wave (SFCW) radar, generating spectrograms of multiple frequencies which introduce different scattering properties and frequency resolutions. In the designed parallelism LSTM framework, multiple parallel LSTM sub-networks are trained separately to extract different temporal features from the spectrogram of each frequency and produce corresponding classification probabilities. At the decision level, the probabilities of activity classification from these sub-networks are fused by addition as the recognition output. To validate the proposed method, an experimental data set is collected by using an SFCW radar to monitor 11 participants who continuously perform six activities in sequence with three different transitions and random durations. The validation results demonstrate that the average accuracies of the designed parallelism unidirectional LSTM (Uni-LSTM) and bidirectional LSTM (Bi-LSTM) based on five frequency spectrograms are 85.41% and 96.15%, respectively, outperforming traditional Uni-LSTM and Bi-LSTM networks with only a single-frequency spectrogram by 5.35% and 6.33% at least. Additionally, the recognition accuracy of the parallelism LSTM network reveals an upward trend as the number of multi-frequency spectrograms (namely the number of LSTM subnetworks) increases, and tends to be stable when the number reaches 4.

Continuous Arm Motion Recognition Using Two-stream Spatial-Temporal Neural Network Based on Millimeter Wave Sensor

A RD-T Network for Hand Gesture Recognition Based on Millimeter-Wave Sensor

Human Motion Recognition With Spatial-Temporal-ConvLSTM Network Using Dynamic Range-Doppler Frames Based on Portable FMCW Radar

Two-Stream Time Sequential Network Based Hand Gesture Recognition Method Using Radar Sensor

Continuous Human Activity Recognition With Distributed Radar Sensor Networks and CNN–RNN Architectures

Continuous Human Motion Recognition with a Dynamic Range-Doppler Trajectory Method Based on FMCW Radar

Wireless Hand Gesture Recognition Based on Continuous-Wave Doppler Radar Sensors

Learning to Track Multiple Radar Targets with Long Short-Term Memory Networks

Robust Dynamic Hand Gesture Recognition Based on Millimeter Wave Radar Using Atten-TsNN

Continuous Human Motion Recognition Using Micro-Doppler Signatures in the Scenario with Micro Motion Interference

Temporal-Range-Doppler Features Interpretation And Recognition Of Hand Gestures Using Mmw Fmcw Radar Sensors

A Robust Hand Gesture Sensing and Recognition Based on Dual-Flow Fusion with FMCW Radar.

A Convolutional Neural Network for Human Motion Recognition and Classification Using a Millimeter-Wave Doppler Radar

Human Motion Recognition Exploiting Radar with Stacked Recurrent Neural Network

Human Motion Recognition by Micro-Doppler Features and Concatenated CNN-LSTM Network

Micro-motion Gesture Recognition Based on Multi-channel Frequency Modulated Continuous Wave Millimeter Wave Radar

Hand Gesture Recognition Method Based on Dual-Channel Convolutional Neural Network

Automatic Arm Motion Recognition Based on Radar Micro-Doppler Signature Envelopes

Continuous Human Activity Recognition through Parallelism LSTM with Multi-Frequency Spectrograms

Long-Range Gesture Recognition Using Millimeter Wave Radar

A Multi-feature Fusion Temporal Neural Network for Multi-hand Gesture Recognition Using Millimeter-wave Radar Sensor