Abstract:Human activity recognition (HAR) based on wearable sensors has been widely used in various fields such as health monitoring, healthcare, and fitness due to its portability, accuracy, and real-time capabilities. Currently, advanced technologies involve converting time-series into images and deep learning for recognition, which addresses issues of subjectivity and dependence on data quality inherent in traditional processing methods. However, each of the current methods for converting time-series data into images only focuses on one type of feature representation such that one of these methods is insufficient to fully characterize the data, which results in lower accuracy in activity recognition. To address the above problems, we propose a novel method for cross-modal image representation of time-series and a dual-stream feature enhanced network model, which enables HAR based on a single-node wearable sensor. Firstly, three methods including Markov Transition Field (MTF), Recurrence Plot (RP), and Gramian Angular Field (GAF), are employed to encode the time-series into three channels (R, G, B) of color images, facilitating the fusion of features such as amplitude variation, nonlinearity, and local temporal relationship. Then, a multi-channel CNN with a Global Attention Mechanism (GAM) is employed in the model to process images, which captures both channel and spatial information. A network combining CNN and Long Short-Term Memory (LSTM) with Self-Attention Mechanism (SA) is utilized to process time-series, which captures temporal features. Simultaneously, the introduction of residual structures makes the network easy to train, thereby enhancing overall performance. The experimental results demonstrate that the proposed model can accurately identify six different gymnastic activities, achieving an accuracy rate of 99.40%. This study provides a new direction for processing time-series and offers better applications in the field of HAR based on wearable sensors.

Triple Cross-Domain Attention on Human Activity Recognition Using Wearable Sensors

Multihead-Res-SE Residual Network with Attention for Human Activity Recognition

Attention-based LSTM Network for Wearable Human Activity Recognition

Attention-Based Sensor Fusion for Human Activity Recognition Using IMU Signals

A Novel Deep Multifeature Extraction Framework Based on Attention Mechanism Using Wearable Sensor Data for Human Activity Recognition

Innovative Dual-Decoupling CNN with Layer-wise Temporal-Spatial Attention for Sensor-Based Human Activity Recognition

DanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensors

Privacy-preserving Activity Recognition Using Multimodal Sensors in Smart Office

TCN-attention-HAR: human activity recognition based on attention mechanism time convolutional network

Large Receptive Field Attention: An Innovation in Decomposing Large-Kernel Convolution for Sensor-Based Activity Recognition

An Information Gain-Based Model and an Attention-Based RNN for Wearable Human Activity Recognition

CIR-DFENet: Incorporating Cross-Modal Image Representation and Dual-Stream Feature Enhanced Network for Activity Recognition

Multi-STMT: Multi-Level Network for Human Activity Recognition Based on Wearable Sensors

Cross-Attention Enhanced Pyramid Multi-Scale Networks for Sensor-based Human Activity Recognition

Plug-and-play Multi-Dimensional Attention Module for Accurate Human Activity Recognition

A Deep Dilated Convolutional Self-attention Model for Multimodal Human Activity Recognition

A Novel Deep Learning Bi-GRU-I Model for Real-Time Human Activity Recognition Using Inertial Sensors

A Multi-dimensional Parallel Convolutional Connected Network Based on Multi-source and Multi-modal Sensor Data for Human Activity Recognition

MMTSA: Multimodal Temporal Segment Attention Network for Efficient Human Activity Recognition

Channel Attention for Sensor-based Activity Recognition: Embedding Features into All Frequencies in DCT Domain

A Multidimensional Parallel Convolutional Connected Network Based on Multisource and Multimodal Sensor Data for Human Activity Recognition