Abstract:The popularity and diffusion of wearable devices provides new opportunities for sensor-based human activity recognition that leverages deep learning-based algorithms. Although impressive advances have been made, two major challenges remain. First, sensor data is often incomplete or noisy due to sensor placement and other issues as well as data transmission failure, calling for imputation of missing values, which also introduces noise. Second, human activity has multi-scale characteristics. Thus, different groups of people and even the same person may behave differently under different circumstances. To address these challenges, we propose a multilevel heterogeneous neural network, called MHNN, for sensor data analysis. We utilize multilevel discrete wavelet decomposition to extract multi-resolution features from sensor data. This enables distinguishing signals with different frequencies, thereby suppressing noise. As the components resulting from the decomposition are heterogeneous, we equip the proposed model with heterogeneous feature extractors that enable the learning of multi-scale features. Due to the complementarity of these features, we also include a cross aggregation module for enhancing their interactions. An experimental study using seven publicly available datasets offers evidence that MHNN can outperform other cutting-edge models and offers evidence of robustness to missing values and noise. An ablation study confirms the importance of each module.

What problem does this paper attempt to address?

### Main Problems Addressed by the Paper This paper primarily addresses the issue of human activity recognition (HAR) using wearable device sensor data, proposing a novel Multi-level Heterogeneous Network (MHNN) to tackle two main challenges: 1. **Incomplete and Noisy Data**: Due to sensor placement, transmission failures, and other reasons, sensor data often contain missing values or noise. These quality issues can affect the performance of deep learning-based methods and lead to prediction errors. 2. **Multi-scale Characteristics**: Different individuals may exhibit different patterns when performing the same activity, and the same individual may behave differently in different contexts. This requires the recognition model to handle features at different time scales. ### Solution Overview To address the above challenges, the paper proposes a Multi-level Heterogeneous Neural Network (MHNN) based on Multi-level Discrete Wavelet Decomposition (MDWD). The main features of this approach include: - **Multi-resolution Feature Extraction**: Using multi-level discrete wavelet decomposition to extract features at different resolutions from the raw signal, effectively suppressing noise. - **Heterogeneous Feature Extractors**: Considering that wavelet decomposition results at different levels contain different amounts of information, feature extraction subnets of varying complexity are designed to adapt to different levels of components. - **Feature Fusion Module**: To enhance the interaction between multi-level features, a cross-aggregation module is introduced, thereby improving feature representation capabilities. ### Experimental Validation Experiments were conducted on seven public datasets, and the results show that MHNN outperforms 28 other methods based on handcrafted features or deep learning in terms of recognition accuracy. Additionally, the paper demonstrates the robustness of the proposed method to noise and missing values and validates the impact of model parameters through sensitivity analysis. Finally, the effectiveness of each component is verified through ablation studies.

Disentangling Imperfect: A Wavelet-Infused Multilevel Heterogeneous Network for Human Activity Recognition in Flawed Wearable Sensor Data

Wearable Sensor Based Multimodal Human Activity Recognition Exploiting the Diversity of Classifier Ensemble.

A Multidimensional Parallel Convolutional Connected Network Based on Multisource and Multimodal Sensor Data for Human Activity Recognition

A Multi-dimensional Parallel Convolutional Connected Network Based on Multi-source and Multi-modal Sensor Data for Human Activity Recognition

Wearable Multi-Sensor Data Fusion Approach for Human Activity Recognition Using Machine Learning Algorithms

Multiscale Deep Feature Learning for Human Activity Recognition Using Wearable Sensors

Deep Wavelet Convolutional Neural Networks for Multimodal Human Activity Recognition Using Wearable Inertial Sensors

Wearable Sensor-Based Human Activity Recognition Method with Multi-Features Extracted from Hilbert-Huang Transform.

Wearable Sensor-Based Residual Multifeature Fusion Shrinkage Networks for Human Activity Recognition

Multi-STMT: Multi-Level Network for Human Activity Recognition Based on Wearable Sensors

Imaging and Fusing Time Series for Wearable Sensor-Based Human Activity Recognition

HHAR-net: Hierarchical Human Activity Recognition using Neural Networks

A Novel Distribution-Embedded Neural Network for Sensor-Based Activity Recognition

Human activity recognition using wearable sensors by heterogeneous convolutional neural networks

Deep Learning for Heterogeneous Human Activity Recognition in Complex IoT Applications

Multimodal Human Activity Recognition From Wearable Inertial Sensors Using Machine Learning

Human Activity Recognition using Continuous Wavelet Transform and Convolutional Neural Networks

A Robust Deep Feature Extraction Method for Human Activity Recognition Using a Wavelet Based Spectral Visualisation Technique

A Novel Deep Multifeature Extraction Framework Based on Attention Mechanism Using Wearable Sensor Data for Human Activity Recognition

Deep learning based multimodal complex human activity recognition using wearable devices

A multi-graph convolutional network based wearable human activity recognition method using multi-sensors