Abstract:Numerous methods and applications have been proposed in human activity recognition (HAR). This paper presents a mini-survey of recent HAR studies and our originally developed benchmark datasets of two types using environmental sensors. For the first dataset, we specifically examine human pose estimation and slight motion recognition related to activities of daily living (ADL). Our proposed method employs OpenPose. It describes feature vectors without effects of objects or scene features, but with a convolutional neural network (CNN) with the VGG-16 backbone, which recognizes behavior patterns after classifying the obtained images into learning and verification subsets. The first dataset comprises time-series panoramic images obtained using a fisheye lens monocular camera with a wide field of view. We attempted to recognize five behavior patterns: eating, reading, operating a smartphone, operating a laptop computer, and sitting. Even when using panoramic images including distortions, results demonstrate the capability of recognizing properties and characteristics of slight motions and pose-based behavioral patterns. The second dataset was obtained using five environmental sensors: a thermopile sensor, a CO2 sensor, and air pressure, humidity, and temperature sensors. Our proposed sensor system obviates the need for constraint; it also preserves each subject’s privacy. Using a long short-term memory (LSTM) network combined with CNN, which is a deep-learning model dealing with time-series features, we recognized eight behavior patterns: eating, operating a laptop computer, operating a smartphone, playing a game, reading, exiting, taking a nap, and sitting. The recognition accuracy for the second dataset was lower than for the first dataset consisting of images, but we demonstrated recognition of behavior patterns from time-series of weak sensor signals. The recognition results for the first dataset, after accuracy evaluation, can be reused for automatically annotated labels applied to the second dataset. Our proposed method actualizes semi-automatic annotation, false recognized category detection, and sensor calibration. Feasibility study results show the new possibility of HAR used for ADL based on unique sensors of two types.

Human Activity Recognition In A Smart Home Environment With Stacked Denoising Autoencoders

Leveraging Wearable Sensors for Human Daily Activity Recognition with Stacked Denoising Autoencoders

Recognizing Human Activities From Raw Accelerometer Data Using Deep Neural Networks

Discovery and Recognition of Emerging Human Activities Using a Hierarchical Mixture of Directional Statistical Models.

Representation Learning for Minority and Subtle Activities in a Smart Home Environment

Understanding and Improving Deep Neural Network for Activity Recognition

A Mini-Survey and Feasibility Study of Deep-Learning-Based Human Activity Recognition from Slight Feature Signals Obtained Using Privacy-Aware Environmental Sensors

Convolutional Neural Network Bootstrapped by Dynamic Segmentation and Stigmergy-Based Encoding for Real-Time Human Activity Recognition in Smart Homes

Deep Learning for Sensor-based Activity Recognition: A Survey

Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities

Investigating Deep Neural Network Architecture and Feature Extraction Designs for Sensor-based Human Activity Recognition

A Multi-Task Deep Learning Approach for Sensor-based Human Activity Recognition and Segmentation

A Semi-Supervised Transfer Learning with Dynamic Associate Domain Adaptation for Human Activity Recognition Using WiFi Signals

Human activity recognition based on multienvironment sensor data

Know Thy Neighbors: A Graph Based Approach for Effective Sensor-Based Human Activity Recognition in Smart Homes

Deep Auto-Set: A Deep Auto-Encoder-Set Network for Activity Recognition Using Wearables

A Novel Deep Multifeature Extraction Framework Based on Attention Mechanism Using Wearable Sensor Data for Human Activity Recognition

DeepHAR: a deep feed-forward neural network algorithm for smart insole-based human activity recognition

SDHAR-HOME: A Sensor Dataset for Human Activity Recognition at Home

Multi-User Activity Recognition In A Smart Home

Class imbalance in multi-resident activity recognition: an evaluative study on explainability of deep learning approaches