Abstract:Human activity recognition (HAR) based on wearable sensors has been widely used in various fields such as health monitoring, healthcare, and fitness due to its portability, accuracy, and real-time capabilities. Currently, advanced technologies involve converting time-series into images and deep learning for recognition, which addresses issues of subjectivity and dependence on data quality inherent in traditional processing methods. However, each of the current methods for converting time-series data into images only focuses on one type of feature representation such that one of these methods is insufficient to fully characterize the data, which results in lower accuracy in activity recognition. To address the above problems, we propose a novel method for cross-modal image representation of time-series and a dual-stream feature enhanced network model, which enables HAR based on a single-node wearable sensor. Firstly, three methods including Markov Transition Field (MTF), Recurrence Plot (RP), and Gramian Angular Field (GAF), are employed to encode the time-series into three channels (R, G, B) of color images, facilitating the fusion of features such as amplitude variation, nonlinearity, and local temporal relationship. Then, a multi-channel CNN with a Global Attention Mechanism (GAM) is employed in the model to process images, which captures both channel and spatial information. A network combining CNN and Long Short-Term Memory (LSTM) with Self-Attention Mechanism (SA) is utilized to process time-series, which captures temporal features. Simultaneously, the introduction of residual structures makes the network easy to train, thereby enhancing overall performance. The experimental results demonstrate that the proposed model can accurately identify six different gymnastic activities, achieving an accuracy rate of 99.40%. This study provides a new direction for processing time-series and offers better applications in the field of HAR based on wearable sensors.

Multihead-Res-SE Residual Network with Attention for Human Activity Recognition

Multi-STMT: Multi-Level Network for Human Activity Recognition Based on Wearable Sensors

A Multi-dimensional Parallel Convolutional Connected Network Based on Multi-source and Multi-modal Sensor Data for Human Activity Recognition

DanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensors

Wearable Sensor-Based Residual Multifeature Fusion Shrinkage Networks for Human Activity Recognition

Sensors-based Human Activity Recognition with Convolutional Neural Network and Attention Mechanism

Human Activity Recognition using A Improved Model Based on Multi-Head CNN-LSTM

A Multidimensional Parallel Convolutional Connected Network Based on Multisource and Multimodal Sensor Data for Human Activity Recognition

Attention-based LSTM Network for Wearable Human Activity Recognition

A Novel Deep Multifeature Extraction Framework Based on Attention Mechanism Using Wearable Sensor Data for Human Activity Recognition

Cross-Attention Enhanced Pyramid Multi-Scale Networks for Sensor-based Human Activity Recognition

Innovative Dual-Decoupling CNN with Layer-wise Temporal-Spatial Attention for Sensor-Based Human Activity Recognition

Multi-level Contrast Network for Wearables-based Joint Activity Segmentation and Recognition.

Human Activity Recognition Based on Gramian Angular Field and Deep Convolutional Neural Network

A Deep Dilated Convolutional Self-attention Model for Multimodal Human Activity Recognition

CIR-DFENet: Incorporating Cross-Modal Image Representation and Dual-Stream Feature Enhanced Network for Activity Recognition

Deep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors

MAG-Res2Net: a novel deep learning network for human activity recognition

Deep Neural Networks for Sensor-Based Human Activity Recognition Using Selective Kernel Convolution

Human Activity Recognition Based on Self-Attention Mechanism in WiFi Environment

Multi-scale Context-Aware Networks Based on Fragment Association for Human Activity Recognition