Past, Present, and Future of Sensor-based Human Activity Recognition using Wearables: A Surveying Tutorial on a Still Challenging Task

Harish Haresamudram,Chi Ian Tang,Sungho Suh,Paul Lukowicz,Thomas Ploetz

2024-11-12

Abstract:In the many years since the inception of wearable sensor-based Human Activity Recognition (HAR), a wide variety of methods have been introduced and evaluated for their ability to recognize activities. Substantial gains have been made since the days of hand-crafting heuristics as features, yet, progress has seemingly stalled on many popular benchmarks, with performance falling short of what may be considered 'sufficient'-- despite the increase in computational power and scale of sensor data, as well as rising complexity in techniques being employed. The HAR community approaches a new paradigm shift, this time incorporating world knowledge from foundational models. In this paper, we take stock of sensor-based HAR -- surveying it from its beginnings to the current state of the field, and charting its future. This is accompanied by a hands-on tutorial, through which we guide practitioners in developing HAR systems for real-world application scenarios. We provide a compendium for novices and experts alike, of methods that aim at finally solving the activity recognition problem.

Signal Processing,Machine Learning

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is **the challenges of automatic human activity recognition (HAR) in practical applications**, especially how to improve the accuracy and generalization ability of activity recognition. Although wearable devices and sensor technologies have made remarkable progress, the performance of activity recognition on some challenging benchmark tasks still remains stagnant and fails to reach a satisfactory level. Specifically, the paper focuses on the following aspects:

1. **Insufficient data annotation**: The lack of large - scale annotated datasets is a major problem currently faced by HAR research. This has led to the poor generalization ability of existing models across different datasets and users.

2. **Representation learning**: Traditional feature - engineering methods rely on manually - designed features, which often do not have good generalization ability. Therefore, the paper explores how to learn richer representations through self - supervised learning (SSL) and other multi - modal learning methods, thereby improving the performance of HAR systems.

3. **Multi - modal learning**: The paper discusses how to use multi - modal data (such as video, audio, etc.) to enhance the accuracy of activity recognition. By combining multiple data sources, the complexity of activities can be better captured.

4. **Application of base models**: The paper also explores how to introduce base models (such as Transformer) into HAR to take advantage of their powerful representation and context - understanding abilities.

In summary, this paper aims to comprehensively review and summarize the past, present, and future of the field of human activity recognition based on wearable sensors, and provide practical tutorials to help researchers and practitioners develop more effective HAR systems, especially when dealing with complex real - world application scenarios.