Abstract:Weakly supervised temporal action localization aims to locate the temporal boundaries of action instances in untrimmed videos using video-level labels and assign them the corresponding action category. Generally, it is solved by a pipeline called "localization-by-classification", which finds the action instances by classifying video snippets. However, since this approach optimizes the video-level classification objective, the generated activation sequences often suffer interference from class-related scenes, resulting in a large number of false positives in the prediction results. Many existing works treat background as an independent category, forcing models to learn to distinguish background snippets. However, under weakly supervised conditions, the background information is fuzzy and uncertain, making this method extremely difficult. To alleviate the impact of false positives, we propose a new actionness-guided false positive suppression framework. Our method seeks to suppress false positive backgrounds without introducing the background category. Firstly, we propose a self-training actionness branch to learn class-agnostic actionness, which can minimize the interference of class-related scene information by ignoring the video labels. Secondly, we propose a false positive suppression module to mine false positive snippets and suppress them. Finally, we introduce the foreground enhancement module, which guides the model to learn the foreground with the help of the attention mechanism as well as class-agnostic actionness. We conduct extensive experiments on three benchmarks (THUMOS14, ActivityNet1.2, and ActivityNet1.3). The results demonstrate the effectiveness of our method in suppressing false positives and it achieves the state-of-the-art performance. Code: https://github.com/lizhilin-ustc/AFPS.

Weakly-Supervised Action Recognition and Localization via Knowledge Transfer.

TwinNet: Twin Structured Knowledge Transfer Network for Weakly Supervised Action Localization

Knowledge-guided Pre-Training and Fine-Tuning: Video Representation Learning for Action Recognition

Weakly-supervised action localization via embedding-modeling iterative optimization

Entropy Guided Attention Network for Weakly-Supervised Action Localization.

UntrimmedNets for Weakly Supervised Action Recognition and Detection

Transferable Knowledge-Based Multi-Granularity Fusion Network for Weakly Supervised Temporal Action Detection

Weakly-supervised Temporal Action Localization with Adaptive Clustering and Refining Network

Annotation-Efficient Untrimmed Video Action Recognition

Cross-Video Contextual Knowledge Exploration and Exploitation for Ambiguity Reduction in Weakly Supervised Temporal Action Localization

Weakly-Supervised Action Localization by Hierarchically-structured Latent Attention Modeling

Learning to Localize Actions from Moments

Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation

Action-Semantic Consistent Knowledge for Weakly-Supervised Action Localization

Weakly Supervised Temporal Action Localization Using Deep Metric Learning

Efficient Action Detection in Untrimmed Videos via Multi-Task Learning

Modeling Sub-Actions for Weakly Supervised Temporal Action Localization

Weakly supervised temporal action localization with actionness-guided false positive suppression

Action Shuffling for Weakly Supervised Temporal Localization

A Joint Model for Action Localization and Classification in Untrimmed Video with Visual Attention

Completeness Modeling and Context Separation for Weakly Supervised Temporal Action Localization