Abstract:The recent advances in 3D Convolutional Neural Networks (3D CNNs) have shown promising performance for untrimmed video action detection, employing the popular detection framework that heavily relies on the temporal action proposal generations as the input of the action detector and localization regressor. In practice the proposals usually contain strong intra and inter relations among them, mainly stemming from the temporal and spatial variations in the video actions. However, most of existing 3D CNNs ignore the relations and thus suffer from the redundant proposals degenerating the detection performance and efficiency. To address this problem, we propose graph attention based proposal 3D ConvNets (AGCN-P-3DCNNs) for video action detection. Specifically. our proposed graph attention is composed of intra attention based GCN and inter attention based GCN. We use intra attention to learn the intra long-range dependencies inside each action proposal and update node matrix of Intra Attention based GCN, and use inter attention to learn the inter dependencies between different action proposals as adjacency matrix of Inter Attention based GCN. Afterwards, we fuse intra and inter attention to model intra long-range dependencies and inter dependencies simultaneously. Another contribution is that we propose a simple and effective framewise classifier. which enhances the feature presentation capabilities of backbone model. Experiments on two proposal 3D ConvNets based models (P-C3D and P-ResNet) and two popular action detection benchmarks (THUMOS 2014, ActivityNet v1.3) demonstrate the state-of-the-art performance achieved by our method. Particularly. P-C3D embedded with our module achieves average mAP 3.7% improvement on THUMOS 2014 dataset compared to original model.

Spatio-temporal Deformable 3D ConvNets with Attention for Action Recognition

Learning SpatioTemporal and Motion Features in a Unified 2D Network for Action Recognition

Spatio-Temporal Attention Networks for Action Recognition and Detection

A Channel-Wise Spatial-Temporal Aggregation Network for Action Recognition

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Learning Attention-Enhanced Spatiotemporal Representation for Action Recognition

Unified Spatio-Temporal Attention Networks for Action Recognition in Videos.

Spatio-Temporal Adaptive Network with Bidirectional Temporal Difference for Action Recognition

STCA: an action recognition network with spatio-temporal convolution and attention

An efficient attention module for 3d convolutional neural networks in action recognition

Action Recognition by an Attention-Aware Temporal Weighted Convolutional Neural Network.

Bi-direction Hierarchical LSTM with Spatial-Temporal Attention for Action Recognition

Graph Attention Based Proposal 3D ConvNets for Action Detection.

STA-TSN: Spatial-Temporal Attention Temporal Segment Network for Action Recognition in Video.

T-C3D: Temporal Convolutional 3D Network for Real-Time Action Recognition

StNet: Local and Global Spatial-Temporal Modeling for Action Recognition

Spatio-temporal Collaborative Convolution for Video Action Recognition

Integrating Temporal and Spatial Attention for Video Action Recognition

SSTA-Net: Self-supervised Spatio-Temporal Attention Network for Action Recognition.

STH: Spatio-Temporal Hybrid Convolution for Efficient Action Recognition

Spatio-Temporal Attention-Based LSTM Networks for 3D Action Recognition and Detection