Abstract:Despite remarkable progress, temporal action detection is still limited for real application due to the great amount of manual annotations. This issue motivates interest in addressing this task under weak supervision, namely, locating the action instances using only video-level class labels. Many current works on this task are mainly based on the Class Activation Sequence (CAS), which is generated by the video classification network to describe the probability of each snippet being in a specific action class of the video. However, the CAS generated by a simple classification network can only focus on local discriminative parts instead of locating the entire interval of target actions. In this paper, we present a novel framework to handle this issue. Specifically, we propose to utilize convolutional kernels with varied dilation rates to enlarge the receptive fields, which can transfer the discriminative information to the surrounding non-discriminative regions. Then, we design a cascaded module with the proposed Online Adversarial Erasing (OAE) mechanism to further mine more relevant regions of target actions by feeding the erased-feature maps of discovered regions back into the system. In addition, inspired by the transfer learning method, we adopt an additional module to transfer the knowledge from trimmed videos to untrimmed videos to promote the classification performance on untrimmed videos. Finally, we employ a boundary regression module embedded with Outer-Inner-Contrastive (OIC) loss to automatically predict the boundaries based on the enhanced CAS. Extensive experiments are conducted on two challenging datasets, THUMOS14 and ActivityNet-1.3, and the experimental results clearly demonstrate the superiority of our unified framework.

Continuous action recognition with weakly labelling videos

Continuous Action Recognition Based on Hybrid CNN-LDCRF Model

Human Action Recognition From Digital Videos Based on Deep Learning.

Continuous Action Recognition and Segmentation in Untrimmed Videos

Action recognition with temporal scale-invariant deep learning framework

Continuous Human Action Recognition in Real Time

Temporal Distinct Representation Learning for Action Recognition

Continuous Action Segmentation and Recognition Using Hybrid Convolutional Neural Network-Hidden Markov Model Model

Towards Weakly Supervised End-to-end Learning for Long-video Action Recognition

Action Recognition By Learning Deep Multi-Granular Spatio-Temporal Video Representation

Weakly Supervised Energy-Based Learning for Action Segmentation

Integrating Temporal and Spatial Attention for Video Action Recognition

Action Recognition by an Attention-Aware Temporal Weighted Convolutional Neural Network.

DB-LSTM: Densely-connected Bi-directional LSTM for Human Action Recognition

Modeling Sub-Actions for Weakly Supervised Temporal Action Localization

Transferable Knowledge-Based Multi-Granularity Fusion Network for Weakly Supervised Temporal Action Detection

Bi-direction Hierarchical LSTM with Spatial-Temporal Attention for Action Recognition

Unsupervised Deep Learning of Mid-Level Video Representation for Action Recognition.

Action Recognition Using Visual Attention with Reinforcement Learning.

Realistic Human Action Recognition: when CNNS Meet LDS

Learning Motion and Content-Dependent Features with Convolutions for Action Recognition