Abstract:According to the real-living environment, radar-based human activity recognition (HAR) is dedicated to recognizing and classifying a sequence of activities rather than individual activities, thereby drawing more attention in practical applications of security surveillance, health care and human-computer interactions. This paper proposes a parallelism long short-term memory (LSTM) framework with the input of multi-frequency spectrograms to implement continuous HAR. Specifically, frequency-division short-time Fourier transformation (STFT) is performed on the data stream of continuous activities collected by a stepped-frequency continuous-wave (SFCW) radar, generating spectrograms of multiple frequencies which introduce different scattering properties and frequency resolutions. In the designed parallelism LSTM framework, multiple parallel LSTM sub-networks are trained separately to extract different temporal features from the spectrogram of each frequency and produce corresponding classification probabilities. At the decision level, the probabilities of activity classification from these sub-networks are fused by addition as the recognition output. To validate the proposed method, an experimental data set is collected by using an SFCW radar to monitor 11 participants who continuously perform six activities in sequence with three different transitions and random durations. The validation results demonstrate that the average accuracies of the designed parallelism unidirectional LSTM (Uni-LSTM) and bidirectional LSTM (Bi-LSTM) based on five frequency spectrograms are 85.41% and 96.15%, respectively, outperforming traditional Uni-LSTM and Bi-LSTM networks with only a single-frequency spectrogram by 5.35% and 6.33% at least. Additionally, the recognition accuracy of the parallelism LSTM network reveals an upward trend as the number of multi-frequency spectrograms (namely the number of LSTM subnetworks) increases, and tends to be stable when the number reaches 4.

Poster: Attention-based Spatio-Temporal Model for HAR Using Multivariate Time Series.

Multi-STMT: Multi-Level Network for Human Activity Recognition Based on Wearable Sensors

Attention-based LSTM Network for Wearable Human Activity Recognition

A Hierarchical Spatio-Temporal Model for Human Activity Recognition.

Salient Pairwise Spatio-Temporal Interest Points for Real-Time Activity Recognition.

Privacy-preserving Activity Recognition Using Multimodal Sensors in Smart Office

Beyond Isolated Frames: Enhancing Sensor-Based Human Activity Recognition through Intra- and Inter-Frame Attention

Deep Dilation on Multimodality Time Series for Human Activity Recognition.

DanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensors

Bi-STAN: bilinear spatial-temporal attention network for wearable human activity recognition

Human Activity Recognition based on Dynamic Spatio-Temporal Relations

Multihead-Res-SE Residual Network with Attention for Human Activity Recognition

A Deep Dilated Convolutional Self-attention Model for Multimodal Human Activity Recognition

Unified Spatio-Temporal Attention Models for Advanced Human Action Recognition & Detection

MMTSA: Multimodal Temporal Segment Attention Network for Efficient Human Activity Recognition

GRU with Dual Attentions for Sensor-Based Human Activity Recognition

Spatiotemporal Attention for Multivariate Time Series Prediction and Interpretation

Human action recognition in immersive virtual reality based on multi‐scale spatio‐temporal attention network

A Novel Deep Multifeature Extraction Framework Based on Attention Mechanism Using Wearable Sensor Data for Human Activity Recognition

Continuous Human Activity Recognition through Parallelism LSTM with Multi-Frequency Spectrograms

A spatiotemporal multi-feature extraction framework with space and channel based squeeze-and-excitation blocks for human activity recognition