Abstract:Numerous methods and applications have been proposed in human activity recognition (HAR). This paper presents a mini-survey of recent HAR studies and our originally developed benchmark datasets of two types using environmental sensors. For the first dataset, we specifically examine human pose estimation and slight motion recognition related to activities of daily living (ADL). Our proposed method employs OpenPose. It describes feature vectors without effects of objects or scene features, but with a convolutional neural network (CNN) with the VGG-16 backbone, which recognizes behavior patterns after classifying the obtained images into learning and verification subsets. The first dataset comprises time-series panoramic images obtained using a fisheye lens monocular camera with a wide field of view. We attempted to recognize five behavior patterns: eating, reading, operating a smartphone, operating a laptop computer, and sitting. Even when using panoramic images including distortions, results demonstrate the capability of recognizing properties and characteristics of slight motions and pose-based behavioral patterns. The second dataset was obtained using five environmental sensors: a thermopile sensor, a CO2 sensor, and air pressure, humidity, and temperature sensors. Our proposed sensor system obviates the need for constraint; it also preserves each subject’s privacy. Using a long short-term memory (LSTM) network combined with CNN, which is a deep-learning model dealing with time-series features, we recognized eight behavior patterns: eating, operating a laptop computer, operating a smartphone, playing a game, reading, exiting, taking a nap, and sitting. The recognition accuracy for the second dataset was lower than for the first dataset consisting of images, but we demonstrated recognition of behavior patterns from time-series of weak sensor signals. The recognition results for the first dataset, after accuracy evaluation, can be reused for automatically annotated labels applied to the second dataset. Our proposed method actualizes semi-automatic annotation, false recognized category detection, and sensor calibration. Feasibility study results show the new possibility of HAR used for ADL based on unique sensors of two types.

MAGNETO: Edge AI for Human Activity Recognition -- Privacy and Personalization

Cloud-Edge Collaborative Human Activity Recognition System Design and Implementation.

Practical Insights on Incremental Learning of New Human Physical Activity on the Edge

On Handling Catastrophic Forgetting for Incremental Learning of Human Physical Activity on the Edge

Privacy-preserving Activity Recognition Using Multimodal Sensors in Smart Office

Personalized human activity recognition using deep learning and edge-cloud architecture

Privacy-Preserving Machine Learning Based Data Analytics on Edge Devices

HAR-Net:Fusing Deep Representation and Hand-crafted Features for Human Activity Recognition

Enabling Edge Intelligence for Activity Recognition in Smart Homes

On-Device Training Empowered Transfer Learning For Human Activity Recognition

Continual learning in sensor-based human activity recognition: An empirical benchmark analysis

Intelligence at the IoT Edge: Activity Recognition with Low-Power Microcontrollers and Convolutional Neural Networks

MMPoint-GNN: Graph Neural Network with Dynamic Edges for Human Activity Recognition Through a Millimeter-Wave Radar

On-Device Personalization for Human Activity Recognition on STM32

A privacy-enhanced human activity recognition using GAN & entropy ranking of microaggregated data

MAG-Res2Net: a novel deep learning network for human activity recognition

Towards Sustainable Personalized On-Device Human Activity Recognition with TinyML and Cloud-Enabled Auto Deployment

A Mini-Survey and Feasibility Study of Deep-Learning-Based Human Activity Recognition from Slight Feature Signals Obtained Using Privacy-Aware Environmental Sensors

Human Activity Recognition in Mobile Edge Computing: A Low-Cost and High-Fidelity Digital Twin Approach With Deep Reinforcement Learning

Millimeter Wave Radar-based Human Activity Recognition for Healthcare Monitoring Robot

Enhanced federated recognition mechanism based on spatial-temporal model with split learning for multi-view human activity classification in edge intelligent network