Behavior recognition based on the improved density clustering and context-guided Bi-LSTM model
Tongchi Zhou,Aimin Tao,Liangfeng Sun,Boyang Qu,Yanzhao Wang,Hu Huang
DOI: https://doi.org/10.1007/s11042-023-15501-y
IF: 2.577
2023-05-04
Multimedia Tools and Applications
Abstract:Context information is vital to research video human behavior recognition. Under the LSTM together with CNN of the framework, a novel action recognition method, which extracts keyframes by the improved density clustering, and learns Spatio-temporal context information by Context-Guided BiLSTM, is proposed. Specifically, keyframes are firstly extracted by the Gini-based density clustering, then used as the inputs of CNN. Secondly, a deep Spatio-temporal Bi-directional long short-term memory neural network named by Context-Guided BiLSTM, which is built by each Bi-directional LSTM block, is utilized to model temporal dependencies of spatial features. After learning by ConvLSTM and Context-Guided BiLSTM, the results generated by the fusion module are treated as the inputs of the Softmax layer for action recognition. On the three benchmark datasets, UCF sport, UCF11, and jHMDB, experimental results show that our approach achieves good recognition results. The recognition rate is better than that of most existing action recognition methods.
computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering