Abstract:Human activity recognition (HAR) plays a crucial role in human-computer interaction, smart home, health monitoring and elderly care. However, existing methods typically utilize camera, radio frequency (RF) signals or wearable devices for activity recognition. Each single-sensor modality has its inherent limitations, like camera-based methods having blind spots, Wi-Fi-based methods depending on the environment and the inconvenience of wearing Inertial Measurement Unit (IMU) devices. In this paper, we propose a HAR system that leverages three types of sensor combinations: Wi-Fi, IMU and a hybrid of Wi-Fi+IMU. We utilize the Channel State Information (CSI) provided by Wi-Fi and the accelerometer and gyroscope data from IMU devices to capture activity characteristics. Then, we employ six machine learning algorithms to recognize eight types of daily activities. These algorithms include Support Vector Machine (SVM), Multi-layer Perceptron (MLP), Decision Tree, Random Forest, Logistic Regression and k-Nearest Neighbors (kNN). Additionally, we investigate the accuracy of hand gesture recognition using different sensor combinations and analyze the calculation speed of each combination. We conduct a survey to collect user feedback on the performance of various sensor combinations in our HAR system. The results show that the combination of CSI+IMU yields the best HAR recognition accuracy, with a accuracy of 89.38%. The SVM algorithm consistently performs well across all systems, especially excelling in the CSI+IMU system supported by energy and average Fast Fourier Transform (FFT) features. However, we also find that the success of sensor fusion depends on specific algorithms and features. Fusion of CSI and IMU does not universally enhance recognition accuracy for all features and algorithms and can, in some cases, actually reduce accuracy.

HAR-Net: Fusing Deep Representation and Hand-crafted Features for Human Activity Recognition.

HAR-Net:Fusing Deep Representation and Hand-crafted Features for Human Activity Recognition

Attention-based LSTM Network for Wearable Human Activity Recognition

HAR-DeepConvLG: Hybrid Deep Learning-Based Model for Human Activity Recognition in IoT Applications.

Hybrid deep learning approaches for smartphone sensor-based human activity recognition

Human Activity Recognition Using Attention-Mechanism-Based Deep Learning Feature Combination

Wearable Sensor-Based Residual Multifeature Fusion Shrinkage Networks for Human Activity Recognition

Ensemble of deep learning techniques to human activity recognition using smart phone signals

Privacy-preserving Activity Recognition Using Multimodal Sensors in Smart Office

A Multi-dimensional Parallel Convolutional Connected Network Based on Multi-source and Multi-modal Sensor Data for Human Activity Recognition

Deep Learning Models for Real-time Human Activity Recognition with Smartphones

Multi-STMT: Multi-Level Network for Human Activity Recognition Based on Wearable Sensors

A Multidimensional Parallel Convolutional Connected Network Based on Multisource and Multimodal Sensor Data for Human Activity Recognition

A Novel Deep Multifeature Extraction Framework Based on Attention Mechanism Using Wearable Sensor Data for Human Activity Recognition

Attention-Based Deep Learning Framework for Human Activity Recognition with User Adaptation

Multihead-Res-SE Residual Network with Attention for Human Activity Recognition

A Novel Deep Learning Bi-GRU-I Model for Real-Time Human Activity Recognition Using Inertial Sensors

Deep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors

A Fast And Robust Deep Convolutional Neural Networks For Complex Human Activity Recognition Using Smartphone

Human Activity Recognition via Wi-Fi and Inertial Sensors With Machine Learning

Temporal-Spatial Dynamic Convolutional Neural Network for Human Activity Recognition Using Wearable Sensors