MIE-Net: Motion Information Enhancement Network for Fine-Grained Action Recognition Using RGB Sensors
Yutong Li,Miao Ma,Jie Wu,Kaifang Yang,Zhao Pei,Jie Ren
DOI: https://doi.org/10.1109/jsen.2024.3363042
IF: 4.3
2024-01-01
IEEE Sensors Journal
Abstract:In recent years, action recognition has received widespread attention, which classifies actions by extracting features from kinds of sensor data. However, with the growing difficulty of identifying fine-grained actions, certain methods cannot learn sufficient motion and temporal information. Therefore, an effective information enhancement method is required to reason motion clues in video sequences. This article proposes an end-to-end video action recognition framework called the motion information enhancement network (MIE-Net), which consists of two innovative components. The first component, the adaptive fusion module (AFM), selectively extracts the relationships between original and motion-enhanced features to enhance the interaction among different feature information. The second component, a double pooling temporal attention module (DPTAM), implements temporal modeling to enhance subtle information during feature extraction. Finally, a standing long jump dataset (SLJD) containing over 1000 videos from 116 participants is collected by sensor camera, which differs from existing datasets in terms of strong background unbiasedness, to evaluate the effectiveness of our model robustly. Experimental results on SLJD, Something-Something v2, and Diving48 datasets demonstrate that the proposed MIE-Net outperforms most state-of-the-art methods. Our code is released at https://github.com/li-stu-998/MIE-Net.
engineering, electrical & electronic,instruments & instrumentation,physics, applied