Abstract:Weakly supervised temporal action detection aims at localizing temporal positions of action instances in untrimmed videos with only action class labels. In general, previous methods individually classify each frame based on the appearance information and the short-term motion information, and then integrate consecutive high-response action frames into entities which serve as detected action instances. However, the long-range temporal dependencies between action frames are not fully utilized, and the detection results are more likely to be trapped in the most discriminative action segments. To alleviate this issue, we propose a novel two-branch (i.e., the coarse detection branch and the refining detection branch) detection framework with learning the long-range temporal dependencies for obtaining more accurate detection results, where only action class labels are required. The coarse detection branch is used to localize the most discriminative segments of action instances based on a typical multi-instance learning paradigm under the supervision of action class labels, whereas the refining detection branch is expected to localize the less discriminative segments of action instances via learning the long-range temporal dependencies between frames based on the proposed Transformer-style architecture and learning strategies. This collaboration mechanism takes full advantage of complementary information from the provided action class labels and the natural temporal dependencies between action frames, forming a more comprehensive solution. Consequently, our method obtains more precise detection results. Expectedly, the proposed method outperforms recent weakly supervised temporal action detection methods on dataset THUMOS14 and ActivityNet measured by mAP@tIoU and AR@AN.

Multi-Scale Structure-Aware Network for Weakly Supervised Temporal Action Detection

A Channel-Wise Spatial-Temporal Aggregation Network for Action Recognition

Multi-scale Dynamic Network for Temporal Action Detection.

MCMNET: Multi-Scale Context Modeling Network for Temporal Action Detection

Two-Stream Completeness Modeling for Weakly Supervised Temporal Action Detection

Movement Enhancement Toward Multi-Scale Video Feature Representation for Temporal Action Detection

Multi-Scale Based Context-Aware Net for Action Detection.

Transferable Knowledge-Based Multi-Granularity Fusion Network for Weakly Supervised Temporal Action Detection

Weakly Supervised Temporal Action Detection with Temporal Dependency Learning

SCALE MATTERS: TEMPORAL SCALE AGGREGATION NETWORK FOR PRECISE ACTION LOCALIZATION IN UNTRIMMED VIDEOS

ACM-Net: Action Context Modeling Network for Weakly-Supervised Temporal Action Localization

SSTA-Net: Self-supervised Spatio-Temporal Attention Network for Action Recognition.

Multi-Dimensional Attention with Similarity Constraint for Weakly-Supervised Temporal Action Localization

Deep Feature Enhancing and Selecting Network for Weakly Supervised Temporal Action Localization

Separately Guided Context-Aware Network for Weakly Supervised Temporal Action Detection

Adaptive Mutual Supervision for Weakly-Supervised Temporal Action Localization

Multi-Branch Spatial-Temporal Network for Action Recognition

Real-Time Action Detection Method Based on Multi-Scale Spatiotemporal Feature

Superframe-Based Temporal Proposals for Weakly Supervised Temporal Action Detection

Cascaded Pyramid Mining Network for Weakly Supervised Temporal Action Localization

Weakly-Supervised Action Localization by Hierarchical Attention Mechanism with Multi-Scale Fusion Strategies