Abstract:Most existing action recognition approaches directly leverage the video-level features to recognize human actions from videos. Although these methods have made remarkable progress, the accuracy is still unsatisfied. When the test video involves complex backgrounds and activities, existing methods usually suffer from a significant drop in accuracy. Human action is inherently a high-level concept. Merely applying a video classification model without a detailed semantic understanding of the video content, e.g., objects, scene context, object motions, object interactions, is inadequate to tackle the challenges for action recognition. Fine-level semantic understanding of videos generates elementary semantic concepts from the raw video data, such as the semantics of objects and background regions. It can be employed to bridge the gap between the raw video data and the high-level concept of human actions. In this work, we leverage dense semantic segmentation masks, which encode rich semantic details, provide extra information for the network training, and improve the performance of action recognition. We propose a novel deep architecture which is named as Dense Semantics-Assisted Convolutional Neural Networks (DSA-CNNs) to effectively utilize dense semantic information of video by a bottom-up attention way in the spatial stream, while by the way of branch fusion in the temporal stream. To verify the effectiveness of our approach, we conduct extensive experiments on publicly available datasets – UCF101, HMDB51, and Kinetics. The experimental results demonstrate that our approach substantially improves existing methods and achieves very competitive performance. It also shows that our approach is superior to other related methods that utilize extra information for action recognition.

Add: Actionness-Pooled Deep-Convolutional Descriptor

Actionness-pooled Deep-convolutional Descriptor for Fine-Grained Action Recognition.

Action Recognition Using Hybrid Feature Descriptor And Vlad Video Encoding

Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptors

Action-Stage Emphasized Spatiotemporal VLAD for Video Action Recognition

SDAN: Stacked Diverse Attention Network for Video Action Recognition.

Dense Semantics-Assisted Networks For Video Action Recognition

Action Recognition with Multi-Scale Trajectory-Pooled 3D Convolutional Descriptors

Discriminative Segment Focus Network for Fine-grained Video Action Recognition

Action recognition using attention-based spatio-temporal VLAD networks and adaptive video sequences optimization

Action Recognition by an Attention-Aware Temporal Weighted Convolutional Neural Network.

Action Recognition By Learning Deep Multi-Granular Spatio-Temporal Video Representation

Human Action Recognition Combining Sequential Dynamic Images and Two-Stream Convolutional Network

Attribute-based supervised deep learning model for action recognition

End-to-end Video-level Representation Learning for Action Recognition

Content-Aware Attention Network For Action Recognition

Deep Convolutional Neural Networks for Human Action Recognition Using Depth Maps and Postures

Two-Stream 3-D Convnet Fusion for Action Recognition in Videos with Arbitrary Size and Length

Deep Convolutional Neural Networks for Action Recognition Using Depth Map Sequences

An Animation-based Augmentation Approach for Action Recognition from Discontinuous Video

Learning Deep Trajectory Descriptor for Action Recognition in Videos Using Deep Neural Networks.