Multiple inputs modeling of hybrid convolutional neural networks for human activity recognition

Yi-Chun Lai,Yao-Chiang Kan,Kai-Cheng Hsu,Hsueh-Chun Lin
DOI: https://doi.org/10.1016/j.bspc.2024.106034
IF: 5.1
2024-02-18
Biomedical Signal Processing and Control
Abstract:Background and objective This study aims to establish a human activity recognition (HAR) modeling process based on hybrid convolutional neural networks (CNNs) to identify multiple inputs for human activity measurement. Method Two open HAR databases, UCI-HAR and Mhealth, were employed for modeling. The original data were converted into three types of time–frequency domain datasets by the fast Fourier, continuous wavelet, and Hilbert-Huang transform methods. Four heterogeneous inputs, including the original and transformed datasets, were fused for the collaborative HAR modeling process that coupled one- and two-dimensional CNNs with residual and squeeze-and-excitation blocks. The authors repeatedly adjusted the model's architecture for each input to explore the optimal model as the sub-structure of the multi-input-HAR model. Furthermore, a hierarchical modeling process was applied to reinforce the models that misjudge similar activities. Results The designed HAR modeling for the UCI-HAR dataset reached accuracies over 0.95 after 5-fold cross-validation, which approved the data without sampling bias and overfitting. With the approaches, the HAR models based on hybrid CNNs trained by the inputs of UCI-HAR and Mhealth could achieve the best accuracies at 0.96. Then, the multi-input-HAR model coupling the optimized HAR models was trained by UCI-HAR-based inputs, but some activities could be misjudged. A hierarchical modeling process further improved the model was to reach the best accuracy at 0.96. Conclusion The proposed multi-input-HAR modeling can allow heterogeneous data fusion of the inputs including the original measurement and the derived datasets. The established models can be applied to recognize various human activities in healthcare programs.
engineering, biomedical
What problem does this paper attempt to address?