Abstract:Weakly supervised temporal action detection uses the extracted appearance and motion features to localize the action segments in untrimmed videos with only action category labels. Most previous methods detect action segments based on temporally local features, and employ the early fusion or the late fusion machine to combine the knowledge of two feature modalities. However, the temporally local features generally lead to incomplete detection results, and the above-mentioned fusion machines cannot fully use the complementary information between different modalities. In this paper, we propose the separately guided context-aware network to exploit the global contexts and sufficiently leverage different modality information to detect action segments. Specifically, we propose to construct graphs by modeling the co-occurrence relations between frames to gather the global contexts. To fully combine the complementary information of two modalities, the separately guided scheme is proposed, which utilizes two graphs for each feature modality to integrate the contexts revealed by the intra-modality and the cross-modality information respectively. This scheme sufficiently enhances frame representations based on two modalities and facilitates the detection of action frames. And we also present the co-occurrence relation learning strategy under weak supervision to better guide graphs in gathering contexts. Extensive experiments on the THUMOS14 dataset and the ActivityNet dataset demonstrate the superior performance of the proposed method. Particularly, the proposed method achieves a mean average precision of 39.1% and 42.0% on the THUMOS14 and the ActivityNet dataset respectively under the IoU threshold 0.5.

Localizing Action Moments with Dual-Guidance Augmented Discriminability

Temporal Action Localization with Enhanced Instant Discriminability

ADM-Loc: Actionness Distribution Modeling for Point-supervised Temporal Action Localization

Dual DETRs for Multi-Label Temporal Action Detection

Non-Local Temporal Difference Network for Temporal Action Detection

Action Co-Localization In An Untrimmed Video By Graph Neural Networks

Stepwise Multi-grained Boundary Detector for Point-Supervised Temporal Action Localization

ContextDet: Temporal Action Detection with Adaptive Context Aggregation

Separately Guided Context-Aware Network for Weakly Supervised Temporal Action Detection

Proposal-Free Temporal Action Detection Via Global Segmentation Mask Learning

SCALE MATTERS: TEMPORAL SCALE AGGREGATION NETWORK FOR PRECISE ACTION LOCALIZATION IN UNTRIMMED VIDEOS

Movement Enhancement Toward Multi-Scale Video Feature Representation for Temporal Action Detection

Efficient Temporal Action Localization with Temporal Attention and Gaussian Weight.

Advancing Temporal Action Localization with a Boundary Awareness Network

Gaussian Temporal Awareness Networks for Action Localization

Dual relation network for temporal action localization

Introducing Gating and Context into Temporal Action Detection

Temporal Action Localization with Cross Layer Task Decoupling and Refinement

Enriching Local and Global Contexts for Temporal Action Localization.

Exploring Action Centers for Temporal Action Localization

PointTAD: Multi-Label Temporal Action Detection with Learnable Query Points