Abstract:Human Activity Recognition(HAR) plays an important role in the field of ubiquitous computing, which can benefit various human-centric applications such as smart homes, health monitoring, and aging systems. Human Activity Recognition mainly leverages smartphones and wearable devices to collect sensory signals labeled with activity annotations and train machine learning models to recognize individuals' activity automatically. In order to deploy the Human Activity Recognition model in real-world scenarios, however, there are two major barriers. Firstly, sensor data and activity labels are traditionally collected using special experimental equipment in a controlled environment, which means fitting models trained with these datasets may result in poor generalization to real-life scenarios. Secondly, existing studies focus on single or a few modalities of sensor readings, which neglect useful information and its relations existing in multimodal sensor data. To tackle these issues, we propose a novel activity recognition model for multimodal sensory data fusion: Marfusion, and an experimental data collection platform for HAR tasks in real-world scenarios: MarSense. Specifically, Marfusion extensively uses a convolution structure to extract sensory features for each modality of the smartphone sensor and then fuse the multimodal features using the attention mechanism. MarSense can automatically collect a large amount of smartphone sensor data via smartphones among multiple users in their natural-used conditions and environment. To evaluate our proposed platform and model, we conduct a data collection experiment in real-life among university students and then compare our Marfusion model with several other state-of-the-art models on the collected datasets. Experimental Results do not only indicate that the proposed platform collected Human Activity Recognition data in the real-world scenario successfully, but also verify the advantages of the Marfusion model compared to existing models in Human Activity Recognition.

Ftmar: A Fusion Transformer Network for Multi-Resident Activity Recognition

IF-ConvTransformer: A Framework for Human Activity Recognition Using IMU Fusion and ConvTransformer

Multi-resolution Fusion Convolutional Network for Open Set Human Activity Recognition

Self-Supervised Multimodal Fusion Transformer for Passive Activity Recognition

Multi-scale Context-Aware Networks Based on Fragment Association for Human Activity Recognition

Transformer With Bidirectional GRU for Nonintrusive, Sensor-Based Activity Recognition in a Multiresident Environment

Human Activity Recognition Based on Multi-Modal Fusion

Wearable Sensor-Based Residual Multifeature Fusion Shrinkage Networks for Human Activity Recognition

A Multiscale Cross-Modal Interactive Fusion Network for Human Activity Recognition Using Wearable Sensors and Smartphones

Multi-Sensor Fusion Based Robot Self-Activity Recognition

Multi-dimensional convolution transformer for group activity recognition

Marfusion: An Attention-Based Multimodal Fusion Model for Human Activity Recognition in Real-World Scenarios

Multiresolution Fusion Convolutional Network for Open Set Human Activity Recognition

TransTM: A Device-Free Method Based on Time-Streaming Multiscale Transformer for Human Activity Recognition

Multi-resident Type Recognition Based on Ambient Sensors Activity.

Segmentation of Multi-Residential Activities Based on Spatial Correlation and FWC-CPD

Multi-User Activity Recognition In A Smart Home

A multi-scale feature extraction fusion model for human activity recognition

A Two-stage Method for Solving Multi-resident Activity Recognition in Smart Environments

A Survey on Multi-Resident Activity Recognition in Smart Environments

IF-ConvTransformer