An AIoT Framework With Multi-modal Frequency Fusion for WiFi-Based Coarse and Fine Activity Recognition

Junxin Chen,Xu Xu,Tingting Wang,Gwanggil Jeon,David Camacho
DOI: https://doi.org/10.1109/JIOT.2024.3400773
IF: 10.6
2024-01-01
IEEE Internet of Things Journal
Abstract:Benefiting from the progresses of sensing and sustainable computing technologies, recent years have witnessed the dramatic progresses of artificial intelligence of things (AIoT). As a typical AIoT application, WiFi-based human activity recognition has increasing popularities in smart homes. However, WiFibased action recognition often has unstable performance due to environmental interference. To this end, a robust deep learning framework called MSF-Net is proposed for coarse and fine activity recognition using channel state information (CSI) information. First, a dual-stream structure incorporating short-time Fourier transform and discrete wavelet transform is developed to highlight abnormal information in the CSI data. Then, a Transformer is employed as the backbone to effectively extract high-level features. In addition, an attention-based fusion branch is designed to enhance cross-model fusion. Experimental results show that MSF-Net achieves Cohens Kappa scores of 91.82%, 69.76%, 85.91%, and 75.66% on the SignFi, Widar3.0, UT-HAR, and NTU-HAR datasets, respectively. These performance records demonstrate the advantages of MSF-Net over existing methods for coarse and fine activity recognition based on WiFi data.
What problem does this paper attempt to address?