Understanding and Improving Channel Attention for Human Activity Recognition by Temporal-Aware and Modality-Aware Embedding

Chaolei Han,Lei Zhang,Yin Tang,Shige Xu,Fuhong Min,Hao Wu,Aiguo Song

DOI: https://doi.org/10.1109/tim.2022.3191653

IF: 5.6

2022-01-01

IEEE Transactions on Instrumentation and Measurement

Abstract:Unlike image data, it is often hard to understand intricate sensor data for human activity, which generally contains heterogeneous sensor modalities from different body positions. The importance of every modality might also vary over time. Recent studies have witnessed the success of channel attention in boosting model performance. To maintain considerably low computational overhead, it utilizes a global pooling operation to squeeze channel information but neglects the importance of temporal-aware and modality-aware (TAMA) information that is very vital for activity recognition. In this article, we propose a novel attention mechanism called TAMA to factorize global pooling operation into a pair of parallel activity feature embedding processes, which is able to simultaneously highlight the varying importance of TAMA information. Extensive ablation experiments verify that our TAMA attention can achieve competitive results on several standard human activity recognition (HAR) benchmarks without incurring an extra computational burden. Moreover, a series of visualizing analysis is provided to show the improved interpretability by telling which temporal steps or which modalities are more determinant, which is in good line with human common intuition.