Abstract:Human activity recognition (HAR) based on wearable sensors has been widely used in various fields such as health monitoring, healthcare, and fitness due to its portability, accuracy, and real-time capabilities. Currently, advanced technologies involve converting time-series into images and deep learning for recognition, which addresses issues of subjectivity and dependence on data quality inherent in traditional processing methods. However, each of the current methods for converting time-series data into images only focuses on one type of feature representation such that one of these methods is insufficient to fully characterize the data, which results in lower accuracy in activity recognition. To address the above problems, we propose a novel method for cross-modal image representation of time-series and a dual-stream feature enhanced network model, which enables HAR based on a single-node wearable sensor. Firstly, three methods including Markov Transition Field (MTF), Recurrence Plot (RP), and Gramian Angular Field (GAF), are employed to encode the time-series into three channels (R, G, B) of color images, facilitating the fusion of features such as amplitude variation, nonlinearity, and local temporal relationship. Then, a multi-channel CNN with a Global Attention Mechanism (GAM) is employed in the model to process images, which captures both channel and spatial information. A network combining CNN and Long Short-Term Memory (LSTM) with Self-Attention Mechanism (SA) is utilized to process time-series, which captures temporal features. Simultaneously, the introduction of residual structures makes the network easy to train, thereby enhancing overall performance. The experimental results demonstrate that the proposed model can accurately identify six different gymnastic activities, achieving an accuracy rate of 99.40%. This study provides a new direction for processing time-series and offers better applications in the field of HAR based on wearable sensors.

Short-Time Activity Recognition With Wearable Sensors Using Convolutional Neural Network

Temporal-Spatial Dynamic Convolutional Neural Network for Human Activity Recognition Using Wearable Sensors

Attention-based LSTM Network for Wearable Human Activity Recognition

A Novel Method for Short-Time Human Activity Recognition Based on Improved Template Matching Technique.

A Human Activity Recognition Model Based on Wearable Sensor

Sequential Human Activity Recognition Based on Deep Convolutional Network and Extreme Learning Machine Using Wearable Sensors

CIR-DFENet: Incorporating Cross-Modal Image Representation and Dual-Stream Feature Enhanced Network for Activity Recognition

Attention-based Convolutional Neural Network for Weakly Labeled Human Activities Recognition with Wearable Sensors

Shallow Convolutional Neural Networks for Human Activity Recognition Using Wearable Sensors

A Novel Multi-Stage Training Approach for Human Activity Recognition From Multimodal Wearable Sensor Data Using Deep Neural Network

Introducing IHARDS-CNN: A Cutting-Edge Deep Learning Method for Human Activity Recognition Using Wearable Sensors

An Improved Deep Convolutional LSTM for Human Activity Recognition Using Wearable Sensors

Deep Convolutional Networks With Tunable Speed–Accuracy Tradeoff for Human Activity Recognition Using Wearables

Human Activity Recognition and Embedded Application Based on Convolutional Neural Network

Human Activity Recognition with Convolutional Neural Netowrks

Imaging and Fusing Time Series for Wearable Sensor-Based Human Activity Recognition

Human Activity Recognition Based on Wireless Electrocardiogram and Inertial Sensors

A Comparison of Wearable Sensor Configuration Methods for Human Activity Recognition Using CNN

Human Activity Recognition Based on Wearable Sensor Using Hierarchical Deep LSTM Networks

Human Activity Recognition Based on Wavelet-CNN Architecture