Abstract:Despite outstanding performance in image recognition, convolutional neural networks (CNNs) do not yet achieve the same impressive results on action recognition in videos. This is partially due to the inability of CNN for modeling long-range temporal structures especially those involving individual action stages that are critical to human action recognition. In this paper, we propose a novel action-stage (ActionS) emphasized spatiotemporal vector of locally aggregated descriptors (ActionS-ST-VLAD) method to aggregate informative deep features across the entire video according to adaptive video feature segmentation and adaptive segment feature sampling (AVFS-ASFS). In our ActionS-ST-VLAD encoding approach, by using AVFS-ASFS, the keyframe features are chosen and the corresponding deep features are automatically split into segments with the features in each segment belonging to a temporally coherent ActionS. Then, based on the extracted keyframe feature in each segment, a flow-guided warping technique is introduced to detect and discard redundant feature maps, while the informative ones are aggregated by using our exploited similarity weight. Furthermore, we exploit an RGBF modality to capture motion salient regions in the RGB images corresponding to action activity. Extensive experiments are conducted on four public benchmarks—HMDB51, UCF101, Kinetics, and ActivityNet for evaluation. Results show that our method is able to effectively pool useful deep features spatiotemporally, leading to the state-of-the-art performance for video-based action recognition.

DA-VLAD: Discriminative Action Vector of Locally Aggregated Descriptors for Action Recognition

Action Recognition with Stacked Fisher Vectors.

Action Recognition with Uncertain VLAD

MA-VLAD: a fine-grained local feature aggregation scheme for action recognition

Towards Good Practices for Action Video Encoding

Agglomerative Clustering and Residual-VLAD Encoding for Human Action Recognition

Action-Stage Emphasized Spatiotemporal VLAD for Video Action Recognition

Action Recognition Using Hybrid Feature Descriptor And Vlad Video Encoding

Good Practices for Learning to Recognize Actions Using FV and VLAD

Compressed Video Action Recognition Using Motion Vector Representation.

Spatio-Temporal Self-Attention Weighted VLAD Neural Network for Action Recognition.

Temporal Distinct Representation Learning for Action Recognition

SA-DVAE: Improving Zero-Shot Skeleton-Based Action Recognition by Disentangled Variational Autoencoders

Realistic Human Action Recognition: when Deep Learning Meets VLAD

Large Margin Dimensionality Reduction for Action Similarity Labeling

Multiple Instance Discriminative Dictionary Learning For Action Recognition

Encoding Spatio-Temporal Distribution by Generalized Vlad for Action Recognition

Spatial-temporal Histograms of Gradients and HOD-VLAD Encoding for Human Action Recognition

An Efficient Motion Visual Learning Method for Video Action Recognition

A Novel Trajectory-VLAD Based Action Recognition Algorithm for Video Analysis.

Action recognition using attention-based spatio-temporal VLAD networks and adaptive video sequences optimization