Abstract:This work presents the solution of the Signal Sleuths team for the 2024 HASCA WEAR challenge. The challenge focuses on detecting 18 workout activities (and the null class) using accelerometer data from 4 wearables - one worn on each limb. Data analysis revealed inconsistencies in wearable orientation within and across participants, leading to exploring novel multi-wearable data augmentation techniques. We investigate three models using a fixed feature set: (i) "raw": using all data as is, (ii) "left-right swapping": augmenting data by swapping left and right limb pairs, and (iii) "upper-lower limb paring": stacking data by using upper-lower limb pair combinations (2 wearables). Our experiments utilize traditional machine learning with multi-window feature extraction and temporal smoothing. Using 3-fold cross-validation, the raw model achieves a macro F1-score of 90.01%, whereas left-right swapping and upper-lower limb paring improve the scores to 91.30% and 91.87% respectively.

What problem does this paper attempt to address?

The paper mainly addresses the following issues: ### Research Background and Objectives - **Background**: With the development of wearable device technology, it has become increasingly important to use the data collected by these devices for automatic activity recognition. This not only helps track exercise progress or gain long-term health insights but also supports adaptive adjustments of intelligent services such as heart rate monitoring based on detected activities. - **Objective**: The goal of the paper is to propose a solution for the 2024 HASCA WEAR Challenge to detect 18 different exercise activities (and one "null" category) using accelerometer data collected from four wearable devices worn on the limbs. ### Specific Problems Addressed - **Data Inconsistency**: Researchers found inconsistencies in the orientation of wearable devices between different participants and between different recordings of the same participant. For example, some participants might wear the devices on their left and right arms or legs in reversed positions. - **Changes in Device Orientation**: Even within different sessions of the same participant, the orientation of the devices might change. These changes can lead to a decline in model performance. - **Data Missing**: Some participants' data might have missing segments that need to be imputed. - **Complexity**: Some exercise activities are similar to or variants of other activities, increasing the difficulty of the classification task. ### Main Contributions - **Data Augmentation Techniques**: To improve the robustness of the model, researchers proposed two new multi-wearable data augmentation techniques—"left-right swap" and "upper-lower limb pairing." These methods aim to enhance the model's ability to counteract orientation inconsistencies by increasing data diversity. - **Feature Extraction and Machine Learning Models**: The study employed multi-resolution feature extraction methods combined with traditional machine learning models (such as CatBoost) for classification. Additionally, several rotation-invariant statistical aggregation methods were explored. - **Post-Processing Techniques**: Including K-fold majority voting, temporal prediction smoothing, and rule-based activity boosting techniques, further improving the model's predictive performance. ### Experimental Results - **Model Performance**: Experiments showed that the "upper-lower limb pairing" technique performed the best, achieving a macro F1 score of 91.87%, outperforming the original data and other methods. - **Feature Analysis**: The signal magnitude vector (SMV) feature performed the worst, while the proposed rotation-invariant statistical aggregation methods showed significant improvement over SMV but were still inferior to the original features containing rotation information. - **Post-Processing Effect**: Temporal prediction smoothing significantly improved performance across all configurations, with absolute gains ranging from 0.3% to 0.9%.

Left-Right Swapping and Upper-Lower Limb Pairing for Robust Multi-Wearable Workout Activity Detection

Contrastive Left-Right Wearable Sensors (IMUs) Consistency Matching for HAR

Classifying Tasks Performed by Electrical Line Workers Using a Wrist-Worn Sensor: A Data Analytic Approach

Automated Patient Handling Activity Recognition for At-Risk Caregivers Using an Unobtrusive Wearable Sensor

Sensor Data Augmentation from Skeleton Pose Sequences for Improving Human Activity Recognition

Human Activity Recognition Using Self-Supervised Representations of Wearable Data

Magnitude and Rotation Invariant Detection of Transportation Modes with Missing Data Modalities

Anytime, Anywhere: Human Arm Pose from Smartwatch Data for Ubiquitous Robot Control and Teleoperation

Multimodal Human Activity Recognition From Wearable Inertial Sensors Using Machine Learning

Towards Stroke Patients' Upper-limb Automatic Motor Assessment Using Smartwatches

Otago Exercises Monitoring for Older Adults by a Single IMU and Hierarchical Machine Learning Models

Active Machine Learning for Heterogeneity Activity Recognition Through Smartwatch Sensors

Advancing Location-Invariant and Device-Agnostic Motion Activity Recognition on Wearable Devices

Leveraging Sound and Wrist Motion to Detect Activities of Daily Living with Commodity Smartwatches

Learning Activities of Daily Living from Unobtrusive Multimodal Wearables: Towards Monitoring Outpatient Rehabilitation

A Real-time Human Pose Estimation Approach for Optimal Sensor Placement in Sensor-based Human Activity Recognition

Outer Product-Based Fusion of Smartwatch Sensor Data for Human Activity Recognition

ALS-HAR: Harnessing Wearable Ambient Light Sensors to Enhance IMU-based Human Activity Recogntion

DS-MS-TCN: Otago Exercises Recognition with a Dual-Scale Multi-Stage Temporal Convolutional Network

A Wearable Inertial Sensor Approach for Locomotion and Localization Recognition on Physical Activity