Abstract:Human activity recognition (HAR) using body-worn sensors is an active research area in human-computer interaction and human activity analysis. The traditional methods use hand-crafted features to classify multiple activities, which is both heavily dependent on human domain knowledge and results in shallow feature extraction. Rapid developments in deep learning have caused most researchers to switch to deep learning methods, which extract features from raw data automatically. Most of the existing works on human activity recognition tasks involve multimodal sensor data, and these networks mainly focus on the top representation extracted from bottom-up feedforward process without reusing other features from bottom layers. In this paper, we present a novel hybrid deep learning network for human activity recognition that also employs multimodal sensor data; however, our proposed model is a ConvLSTM pipeline that makes full use of the information in each layer extracted along the temporal domain. Thus, we propose a dense connection module (DCM) to ensure maximum information flow between the network layers. Furthermore, we employ a multilayer feature aggregation module (MFAM) to extract features along the spatial domain, and we aggregate the features obtained from every convolutional layer according to the importance of features in different spatial locations. The output of the MFAM is input into two LSTM layers to further model the temporal dependencies. Finally, a fully connected layer and a softmax function are used to compute the probability of each class. We demonstrate the effectiveness of our proposed model on two benchmark datasets: Opportunity and UniMiB-SHAR. The results illustrate that our designed network outperforms the state-of-the-art models. We also conduct experiments on efficiency, multimodal fusion and different hyperparameters to analyze our proposed network. Finally, we carry out ablation and visualization experiments to reveal the effectiveness of the two proposed modules.

An Improved Deep Convolutional LSTM for Human Activity Recognition Using Wearable Sensors

Attention-based LSTM Network for Wearable Human Activity Recognition

Deep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors

Sequential Human Activity Recognition Based on Deep Convolutional Network and Extreme Learning Machine Using Wearable Sensors

CIR-DFENet: Incorporating Cross-Modal Image Representation and Dual-Stream Feature Enhanced Network for Activity Recognition

Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition

Temporal-Spatial Dynamic Convolutional Neural Network for Human Activity Recognition Using Wearable Sensors

Human Activity Recognition Based on Wearable Sensor Using Hierarchical Deep LSTM Networks

HAR-DeepConvLG: Hybrid Deep Learning-Based Model for Human Activity Recognition in IoT Applications.

An Improved Algorithm for Human Activity Recognition Using Wearable Sensors

Multi-STMT: Multi-Level Network for Human Activity Recognition Based on Wearable Sensors

An Information Gain-Based Model and an Attention-Based RNN for Wearable Human Activity Recognition

Wearable Sensor-Based Residual Multifeature Fusion Shrinkage Networks for Human Activity Recognition

A Deep Learning Method for Complex Human Activity Recognition Using Virtual Wearable Sensors

A Hybrid Network Based on Dense Connection and Weighted Feature Aggregation for Human Activity Recognition

Human Activity Recognition Based on Gramian Angular Field and Deep Convolutional Neural Network

Human Activity Recognition Based on Residual Network and BiLSTM

A Multi-dimensional Parallel Convolutional Connected Network Based on Multi-source and Multi-modal Sensor Data for Human Activity Recognition

A Multidimensional Parallel Convolutional Connected Network Based on Multisource and Multimodal Sensor Data for Human Activity Recognition

Deep Convolutional Neural Network with RNNs for Complex Activity Recognition Using Wrist-Worn Wearable Sensor Data

Multiscale Deep Feature Learning for Human Activity Recognition Using Wearable Sensors