Action Density Based Frame Sampling for Human Action Recognition in Videos

Jie Lin,Zekun Mu,Tianqing Zhao,Hanlin Zhang,Xinyu Yang,Peng Zhao
DOI: https://doi.org/10.1016/j.jvcir.2022.103740
IF: 2.887
2023-01-01
Journal of Visual Communication and Image Representation
Abstract:In the action recognition, a proper frame sampling method can not only reduce redundant video information, but also improve the accuracy of action recognition. In this paper, an action density based non-isometric frame sampling method, namely NFS, is proposed to discard the redundant video information and sample the rational frames in videos for neural networks to achieve great accuracy on human action recognition, in which action density is introduced in our method to indicate the intensity of actions in videos. Particularly, the action density determination mechanism, focused-clips division mechanism, and reinforcement learning based frame sampling (RLFS) mechanism are proposed in NFS method. Via the evaluations with various neural networks and datasets, our results show that the proposed NFS method can achieve great effectiveness in frame sampling and can assist in achieving better accuracy on action recognition in comparison with existing methods.
What problem does this paper attempt to address?