Abstract:This paper addresses learning and recognition of human behavior models from multimodal observation in a smart home environment. The proposed approach is part of a framework for acquiring a high-level contextual model for human behavior in an augmented environment. A 3-D video tracking system creates and tracks entities (persons) in the scene. Further, a speech activity detector analyzes audio streams coming from head set microphones and determines for each entity, whether the entity speaks or not. An ambient sound detector detects noises in the environment. An individual role detector derives basic activity like “walking” or “interacting with table” from the extracted entity properties of the 3-D tracker. from the derived multimodal observations, different situations like “aperitif” or “presentation” are learned and detected using statistical models (HMMs). The objective of the proposed general framework is two-fold: the automatic offline analysis of human behavior recordings and the online detection of learned human behavior models. To evaluate the proposed approach, several multimodal recordings showing different situations have been conducted. The obtained results, in particular for offline analysis, are very good, showing that multimodality as well as multiperson observation generation are beneficial for situation recognition. Note to Practitioners—This paper was motivated by the problem of automatically recognizing human behavior and interactions in a smart home environment. The smart home environment is equipped with cameras and microphones that permit the observation of human activity in the scene. The objective is first to visualize the perceived human activities (e.g., for videoconferencing or surveillance of elderly people), and then to provide appropriate services based on these activities. We adopt a layered approach for human activity recognition in the environment. The layered framework is motivated by the human perception of human behavior in the scene (white box). The system first recognizes basic activities of individuals, called roles, like “interacting with table” or “walking.” Then, based on the recognized individual roles, group situations like “aperitif,” “presentation,” or “siesta” are recognized. in this paper, we describe an implementation that is based on a 3-D video tracking system, as well as speech activity detection using head set microphones. We evaluated the system for offline (a posteriori) situation classification and online (in scenario) situation recognition. A prototype system has been realized and installed at France Télécom R&D, visualizing current human behavior in the smart home to a distant user using a web interface. An open issue is still the detection of group dynamics and group formation, which is necessary for group situation recognition in (informal) real settings.

Analysis of human behavior recognition algorithms based on acceleration data

Human Behavior Recognition Based on Motion Data Analysis

Discovery and Recognition of Emerging Human Activities Using a Hierarchical Mixture of Directional Statistical Models.

Human Activity Recognition in Smart Environments Employing Margin Setting Algorithm

Detecting Human Behavior Models From Multimodal Observation in a Smart Home

Human Activity Behavioural Pattern Recognition in Smarthome with Long-hour Data Collection

A hybrid and context-aware framework for normal and abnormal human behavior recognition

A Human Activity Recognition System Based on Dynamic Clustering of Skeleton Data

Machine Learning Model for Human Activity Analysis

Recognition of Approximate Motions of Human Based on Micro-Doppler Features

Comparative Analysis of Different Approaches to Human Activity Recognition Based on Accelerometer Signals

Analyzing the Impact of Different Action Primitives in Designing High-Level Human Activity Recognition Systems

Daily Activity Recognition Based on Triaxial Accelerometer of Elderly People

Integrating Abnormal Gait Detection with Activities of Daily Living Monitoring in Ambient Assisted Living: A 3D Vision Approach

A multimodal approach for human activity recognition based on skeleton and RGB data

Real Life Human Movement Realization in Multimodal Group Communication Using Depth Map Information and Machine Learning

Multi-User Activity Recognition In A Smart Home

Decoding Human Activities: Analyzing Wearable Accelerometer and Gyroscope Data for Activity Recognition

An event-based approach to multi-modal activity modeling and recognition

Landmark-Based Methods for Temporal Alignment of Human Motions

Automated Mobility Context Detection with Inertial Signals