ConvBoost: Boosting ConvNets for Sensor-based Activity Recognition

Shuai Shao,Yu Guan,Bing Zhai,Paolo Missier,Thomas Ploetz
DOI: https://doi.org/10.1145/3596234
2023-05-23
Abstract:Human activity recognition (HAR) is one of the core research themes in ubiquitous and wearable computing. With the shift to deep learning (DL) based analysis approaches, it has become possible to extract high-level features and perform classification in an end-to-end manner. Despite their promising overall capabilities, DL-based HAR may suffer from overfitting due to the notoriously small, often inadequate, amounts of labeled sample data that are available for typical HAR applications. In response to such challenges, we propose ConvBoost -- a novel, three-layer, structured model architecture and boosting framework for convolutional network based HAR. Our framework generates additional training data from three different perspectives for improved HAR, aiming to alleviate the shortness of labeled training data in the field. Specifically, with the introduction of three conceptual layers--Sampling Layer, Data Augmentation Layer, and Resilient Layer -- we develop three "boosters" -- R-Frame, Mix-up, and C-Drop -- to enrich the per-epoch training data by dense-sampling, synthesizing, and simulating, respectively. These new conceptual layers and boosters, that are universally applicable for any kind of convolutional network, have been designed based on the characteristics of the sensor data and the concept of frame-wise HAR. In our experimental evaluation on three standard benchmarks (Opportunity, PAMAP2, GOTOV) we demonstrate the effectiveness of our ConvBoost framework for HAR applications based on variants of convolutional networks: vanilla CNN, ConvLSTM, and Attention Models. We achieved substantial performance gains for all of them, which suggests that the proposed approach is generic and can serve as a practical solution for boosting the performance of existing ConvNet-based HAR models. This is an open-source project, and the code can be found at <a class="link-external link-https" href="https://github.com/sshao2013/ConvBoost" rel="external noopener nofollow">this https URL</a>
Machine Learning,Artificial Intelligence,Computer Vision and Pattern Recognition,Human-Computer Interaction
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: **How to alleviate the over - fitting problem in deep learning (DL) for sensor - based human activity recognition (HAR) due to insufficient labeled data**. Specifically, although deep - learning methods can extract high - level features and perform end - to - end classification, in typical HAR applications, the amount of available labeled sample data is often small and insufficient, which may lead to model over - fitting. To address this challenge, the authors propose the ConvBoost framework, aiming to improve the HAR performance based on convolutional networks (ConvNets) by generating additional training data. ### Specific Problems and Solutions 1. **Over - fitting Problem Caused by Insufficient Labeled Data**: - Deep - learning models usually require a large amount of labeled data for training, but in the HAR field, due to high labeling costs, privacy issues, etc., the amount of available labeled data is often limited. - In this case, the model is prone to over - fitting, that is, it performs well on the training set but poorly on the test set or in practical applications. 2. **Limitations of Existing Solutions**: - Some research directions such as self - supervised learning (SSL) and data augmentation have been explored, but these methods mainly rely on unlabeled data or simple data transformations and fail to fully utilize the existing labeled data. 3. **Proposal of the ConvBoost Framework**: - ConvBoost is a three - layer model architecture and boosting framework, aiming to improve HAR performance by generating additional training data from three different perspectives. - The three - layer structure includes: - **Sampling Layer**: Dense sampling is carried out through the Random - Frame (R - Frame) generator. - **Data Augmentation Layer**: Virtual data is synthesized through the Mix - up generator. - **Resilient Layer**: Problematic signals are simulated through Channel - Drop (C - Drop). - These new concept layers and generators are applicable to any type of convolutional network and are designed based on the characteristics of sensor data, aiming to alleviate the problem of insufficient labeled data. ### Experimental Verification The authors conducted experiments on three standard benchmark datasets (Opportunity, PAMAP2, GOTOV) to verify the effectiveness of the ConvBoost framework. The experimental results show that ConvBoost can significantly improve performance on various convolutional network variants (such as vanilla CNN, ConvLSTM, and Attention Models), demonstrating the generality and practicality of this method. ### Summary The core problem of this paper is to solve the over - fitting problem in deep learning for HAR due to insufficient labeled data. The ConvBoost framework enriches the training data from multiple angles by introducing a new data - generation mechanism, thereby improving the robustness and generalization ability of HAR models based on convolutional networks.