Abstract:Fine-grained video action recognition aims to identify minor and discriminative variations among fine categories of actions. While many recent action recognition methods have been proposed to better model spatio-temporal representations, how to model the interactions among discriminative atomic actions to effectively characterize inter-class and intra-class variations has been neglected, which is vital for understanding fine-grained actions. In this work, we devise a Discriminative Segment Focus Network (DSFNet) to mine the discriminability of segment correlations and localize discriminative action-relevant segments for fine-grained video action recognition. Firstly, we propose a hierarchic correlation reasoning (HCR) module which explicitly establishes correlations between different segments at multiple temporal scales and enhances each segment by exploiting the correlations with other segments. Secondly, a discriminative segment focus (DSF) module is devised to localize the most action-relevant segments from the enhanced representations of HCR by enforcing the consistency between the discriminability and the classification confidence of a given segment with a consistency constraint. Finally, these localized segment representations are combined with the global action representation of the whole video for boosting final recognition. Extensive experimental results on two fine-grained action recognition datasets, i.e. , FineGym and Diving48, and two action recognition datasets, i.e. , Kinetics400 and Something-Something, demonstrate the effectiveness of our approach compared with the state-of-the-art methods.

Temporal Sparse Feature Auto-Combination Deep Network for Video Action Recognition.

Learning SpatioTemporal and Motion Features in a Unified 2D Network for Action Recognition

A Channel-Wise Spatial-Temporal Aggregation Network for Action Recognition

Collaborative Spatio-temporal Feature Learning for Video Action Recognition

Temporal Distinct Representation Learning for Action Recognition

Action-Stage Emphasized Spatiotemporal VLAD for Video Action Recognition

Short-Term Action Recognition by 3D Convolutional Neural Network with Pixel-Wise Evidences

Action Recognition By Learning Deep Multi-Granular Spatio-Temporal Video Representation

Temporal Segment Networks: Towards Good Practices for Deep Action Recognition

Action recognition using attention-based spatio-temporal VLAD networks and adaptive video sequences optimization

Temporal-Spatial Mapping for Action Recognition

Efficient spatio-temporal network for action recognition

Spatio-Temporal Adaptive Network with Bidirectional Temporal Difference for Action Recognition

SSTA-Net: Self-supervised Spatio-Temporal Attention Network for Action Recognition.

Attentional Fused Temporal Transformation Network for Video Action Recognition.

End-to-end Video-level Representation Learning for Action Recognition

Discriminative Segment Focus Network for Fine-grained Video Action Recognition

Attention-based Temporal Weighted Convolutional Neural Network for Action Recognition

Temporal Pyramid Pooling-Based Convolutional Neural Network for Action Recognition

A Deep Learning Method for Video‐based Action Recognition

Temporal Segment Networks for Action Recognition in Videos