Abstract:In the field of human action recognition (HAR), two-stream models have been widely employed. In recent years, traditional two-stream network models have disregarded the interframe sequence characteristics of video, resulting in a decrease in model robustness when local sequence information and long-term motion information interact. In light of this, a novel three-stream neural network is proposed by combining the long-term and short-term characteristics of a frame sequence with spatio-temporal information. Initially, the optical flow sequence image frames and RGB image frames in the video are extracted, the optical flow motion information and image space information in the video is obtained, the corresponding time network and space network are entered, and the spatial information is entered into the sequence feature processing network; the three networks are then pretrained. At the conclusion of training, the operation of feature extraction is executed, the features are incorporated with the parallel fusion algorithm by adding weights, and the behavior categories are classified using Multi-Layer Perception. Experimental results on the UCF11, UCF50, and HMDB51 datasets demonstrate that our model effectively integrates the spatial-temporal and frame-sequence information of human actions, resulting in a significant improvement in recognition accuracy. Its classification accuracy on the three datasets was 99.17%, 97.40%, and 96.88%, respectively, significantly enhancing the generalization capability and validity of conventional two-stream or three-stream models.

Multi-scale Spatial-Temporal Integration Convolutional Tube for Human Action Recognition

Learning SpatioTemporal and Motion Features in a Unified 2D Network for Action Recognition

Mutually Reinforced Spatio-Temporal Convolutional Tube for Human Action Recognition.

A Channel-Wise Spatial-Temporal Aggregation Network for Action Recognition

MiCT: Mixed 3D/2D Convolutional Tube for Human Action Recognition

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Multi-scale Spatiotemporal Information Fusion Network for Video Action Recognition

Human Action Recognition Based on Three-Stream Network with Frame Sequence Features

Local-aware spatio-temporal attention network with multi-stage feature fusion for human action recognition

Human Action Recognition Based on Hierarchical Multi-Scale Adaptive Conv-Long Short-Term Memory Network

Human Action Recognition Combining Sequential Dynamic Images and Two-Stream Convolutional Network

Spatio-Temporal Attention Networks for Action Recognition and Detection

Action Recognition with Multi-Scale Trajectory-Pooled 3D Convolutional Descriptors

STCA: an action recognition network with spatio-temporal convolution and attention

Two-Stream 3-D Convnet Fusion for Action Recognition in Videos with Arbitrary Size and Length

Spatio-temporal Collaborative Convolution for Video Action Recognition

Multi-scale residual network model combined with Global Average Pooling for action recognition

Spatio-Temporal Collaborative Module for Efficient Action Recognition

STM: SpatioTemporal and Motion Encoding for Action Recognition.

Spatiotemporal Multi-Task Network for Human Activity Understanding.

Spatial Mask ConvLSTM Network and Intra-Class Joint Training Method for Human Action Recognition in Video.