Abstract:Robust and accurate human activity recognition (HAR) systems are essential to many human-centric services within active assisted living and healthcare facilities. Traditional HAR systems mostly leverage a single sensing modality (e.g., either wearable, vision, or radio frequency sensing) combined with machine learning techniques to recognize human activities. Such unimodal HAR systems do not cope well with real-time changes in the environment. To overcome this limitation, new HAR systems that incorporate multiple sensing modalities are needed. Multiple diverse sensors can provide more accurate and complete information resulting in better recognition of the performed activities. This article presents WiWeHAR—a multimodal HAR system that uses combined Wi-Fi and wearable sensing modalities to simultaneously sense the performed activities. WiWeHAR makes use of standard Wi-Fi network interface cards to collect the channel state information (CSI) and a wearable inertial measurement unit (IMU) consisting of accelerometer, gyroscope, magnetometer sensors to collect the user's local body movements. We compute the time-variant mean Doppler shift (MDS) from the processed CSI data and magnitude from the inertial data for each sensor of the IMU. Thereafter, we separately extract various time- and frequency-domain features from the magnitude data and the MDS. We apply feature-level fusion to combine the extracted features, and finally supervised learning techniques are used to recognize the performed activities. We evaluate the performance of WiWeHAR by using a multimodal human activity data set, which was obtained from 9 participants. Each participant carried out four activities, such as walking, falling, sitting, and picking up an object from the floor. Our results indicate that the proposed multimodal WiWeHAR system outperforms the unimodal CSI, accelerometer, gyroscope, and magnetometer HAR systems and achieves an overall recognition acc-racy of 99.6%–100%.

Wi-Multi: A Three-Phase System for Multiple Human Activity Recognition with Commercial WiFi Devices.

MultiAuth: Enable Multi-User Authentication with Single Commodity WiFi Device

Toward Multi-User Authentication Using WiFi Signals

Wi-Motion: A Robust Human Activity Recognition Using WiFi Signals

Wi-Fi sensing based person identification and activity recognition using two-phase deep learning model

Wi-chase: A WiFi Based Human Activity Recognition System for Sensorless Environments.

Device-Free Multi-Location Human Activity Recognition Using Deep Complex Network

Temporal-Frequency Attention-Based Human Activity Recognition Using Commercial WiFi Devices

WISDOM: Wi-Fi-Based Contactless Multiuser Activity Recognition

An improved Wi-Fi sensing-based human activity recognition using multi-stage deep learning model

Device-free Single-User Activity Recognition Using Diversified Deep Ensemble Learning.

Joint Activity Recognition and Indoor Localization with WiFi Sensing Based on Multi-View Fusion Strategy.

WiWeHAR: Multimodal Human Activity Recognition Using Wi-Fi and Wearable Sensing Modalities

A Robust and Device-Free Daily Activities Recognition System using Wi-Fi Signals

WiMix: A Lightweight Multimodal Human Activity Recognition System Based on WiFi and Vision

A Real-time Object Detection for WiFi CSI-based Multiple Human Activity Recognition

SiMWiSense: Simultaneous Multi-Subject Activity Classification Through Wi-Fi Signals

Device-Free Human Activity Recognition Using Commercial WiFi Devices

Multi-User Activity Recognition In A Smart Home

Learning Human Activities Through Wi-Fi Channel State Information with Multiple Access Points

WiFi-assisted human activity recognition