Abstract:With continued improvements in wireless sensing technology, the notion of the Internet of Things (IoT) has been widely adopted and has become pervasive owing to its broad applications in scenarios such as ambient assisted living, smart healthcare, and smart homes. In that regard, Human Activity Recognition (HAR) is a vital element of intelligent systems to undertake persistent surveillance of human behavior. Due to the omnipresent impact of smartphones in each person's life, smartphone inertial sensors are used as a case study for this research. Most of the conventional approaches regard HAR as a time series classification problem; yet, the accuracy of recognition degrades for heterogeneous sensors. In this paper, we investigate encoding sensory heterogeneous HAR (HHAR) data into three-channel image representation (i.e. RGB), hence treat the HHAR task as an image classification problem. Since present convolutional network models are computationally heavy when deployed in the IoT environment, we propose a lightweight model image encoded HHAR, calledmulti-scale image encoded HHAR (MS-IE-HHAR). The model employs a Hierarchical Multi-scale Extraction (HME) module followed by an Improved Spatial-wise and Channel-wise Attention (ISCA) module to form the main architecture of the model. The HME module is formed by a group of residually connected shuffle group convolutions (SG-Conv) to extract and learn image representations from different receptive fields while reducing the number of network parameters. The ISCA module combines a lightweight spatial-wise attention (SwA) block and an improved channel-wise attention (CwA) module to enable the network to pay instructive attention to spatial correlations as well as channel interdependency information. Finally, two widely available HHAR public datasets (i.e. HHAR UCI, and MHEALTH) were used to evaluate the performance of the proposed models with accuracy over 98% and 99%, respectively, demonstrati-g the model superiority for modeling HAR from heterogeneous data sources.

Learning behavioral context recognition with multi-stream temporal convolutional networks

Learning Visual Context for Group Activity Recognition.

Vehicle Behavior Recognition using Multi-Stream 3D Convolutional Neural Network

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

A Multi-Task Deep Learning Approach for Sensor-based Human Activity Recognition and Segmentation

A Multi-Stream Convolutional Neural Network Framework for Group Activity Recognition

GraphConvLSTM: Spatiotemporal Learning for Activity Recognition with Wearable Sensors.

A Novel Multi-Stage Training Approach for Human Activity Recognition From Multimodal Wearable Sensor Data Using Deep Neural Network

Convolutional Neural Network Bootstrapped by Dynamic Segmentation and Stigmergy-Based Encoding for Real-Time Human Activity Recognition in Smart Homes

Multi-task, multi-label and multi-domain learning with residual convolutional networks for emotion recognition

Cognitive architecture aided by working-memory for self-supervised multi-modal humans recognition

Contextualized Multidimensional Personality Recognition using Combination of Deep Neural Network and Ensemble Learning

Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition

Human Behavior Recognition Based on Multiscale Convolutional Neural Network.

Deep Learning for Heterogeneous Human Activity Recognition in Complex IoT Applications

Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition

Dual-Stream Contrastive Learning for Channel State Information Based Human Activity Recognition

Weakly Supervised Multi-Task Representation Learning for Human Activity Analysis Using Wearables

Dual-Branch Interactive Networks on Multichannel Time Series for Human Activity Recognition

Temporal-Spatial Dynamic Convolutional Neural Network for Human Activity Recognition Using Wearable Sensors