Abstract:Human activity anticipation is a pillar of new interactive multimedia technologies. It enables better human-computer interaction and creative sensory experience in all types of virtual reality-based multimedia applications. Human behaviors are hierarchically constituted in terms of low-level and high-level activities. Fine-grained activities play a substantial role in the early recognition of simple and complex human movements in real-time with low latency, allowing longer predictions in the future. In this paper, a new objective function is proposed to anticipate fine-grained human activities using IMU data, which still suffers from an imbalance problem. Four customized objective surrogate functions are applied in unidirectional and bidirectional recurrent neural networks and compared to efficiently optimize the model considering the loss of individual classes. The experiments on five datasets across proposed loss functions show that the proposed model significantly outperforms the counterpart methods and advances the state-of-the-art. The proposed model shows accuracy scores of up to 98% and 96% for high-level and low-level activities, respectively. Note to Practitioners—There are a large number of methods that can be used for the recognition of human activity in assisted living, smart home technologies, and pervasive healthcare applications. However, studies on these methods tend to focus on recognition rather than anticipation of human activity. Furthermore, they use either vision-based or egocentric video data, which can make it challenging to collect and evaluate in virtual reality-based scenarios. To enable more natural interaction with physical and virtual reality environments, virtual reality-based technologies require built-in algorithms capable of consistently identifying complex and varied human movements. Fine-grained activities are paramount in predicting activities at an early stage to adequately characterize human behaviors. In this paper, a study is conducted to anticipate fine-grained human activity based on IMU (Inertial Measurement Unit) data, which are still subject to the negative effects of an imbalanced sample distribution. The proposed model is evaluated on different benchmarks and compared with baseline methods. The results show that the proposed model outperforms all baselines. These results have important implications for human activity prediction problems.

METAHACI: Meta-learning for Human Activity Classification from IMU Data

Learning-to-Learn Personalised Human Activity Recognition Models

MuJo: Multimodal Joint Feature Space Learning for Human Activity Recognition

Continual learning in sensor-based human activity recognition: An empirical benchmark analysis

MAML MOT: Multiple Object Tracking based on Meta-Learning

Feature Representation and Data Augmentation for Human Activity Classification Based on Wearable IMU Sensor Data Using a Deep LSTM Neural Network

A Recurrent Neural Network Optimization Method for Anticipation of Hierarchical Human Activity

Adaptive Hierarchical Classification for Human Activity Recognition Using Inertial Measurement Unit (IMU) Time-Series Data

An active semi-supervised deep learning model for human activity recognition

HAMLET: A Hierarchical Multimodal Attention-based Human Activity Recognition Algorithm

Meta-HAR: Federated Representation Learning for Human Activity Recognition

Bimodal HAR-An Efficient Approach to Human Activity Analysis and Recognition Using Bimodal Hybrid Classifiers.

Novel hybrid optimization based adaptive deep convolution neural network approach for human activity recognition system

Accuracy Comparison of CNN, LSTM, and Transformer for Activity Recognition Using IMU and Visual Markers

Enhancing Inertial Hand based HAR through Joint Representation of Language, Pose and Synthetic IMUs

Meta-learning meets the Internet of Things: Graph prototypical models for sensor-based human activity recognition

A Deep Learning Approach for Detecting and Classifying Cat Activity to Monitor and Improve Cat’s Well-being using Accelerometer, Gyroscope, and Magnetometer

Sensor Data Acquisition and Multimodal Sensor Fusion for Human Activity Recognition Using Deep Learning

MMTSA: Multimodal Temporal Segment Attention Network for Efficient Human Activity Recognition

Decoding Human Activities: Analyzing Wearable Accelerometer and Gyroscope Data for Activity Recognition