Abstract:This paper addresses the task of temporal sentence grounding (TSG). Although many respectable works have made decent achievements in this important topic, they severely rely on massive expensive video-query paired annotations, which require a tremendous amount of human effort to collect in real-world applications. To this end, in this paper, we target a more practical but challenging TSG setting: unsupervised temporal sentence grounding, where both paired video-query and segment boundary annotations are unavailable during the network training. Considering that some other cross-modal tasks provide many easily available yet cheap labels, we tend to collect and transfer their simple cross-modal alignment knowledge into our complex scenarios: 1) We first explore the entity-aware object-guided appearance knowledge from the paired Image-Noun task, and adapt them into each independent video frame; 2) Then, we extract the event-aware action representation from the paired Video-Verb task, and further refine the action representation into more practical but complicated real-world cases by a newly proposed copy-paste approach; 3) By modulating and transferring both appearance and action knowledge into our challenging unsupervised task, our model can directly utilize this general knowledge to correlate videos and queries, and accurately retrieve the relevant segment without training. Extensive experiments on two challenging datasets (ActivityNet Captions and Charades-STA) show our effectiveness, outperforming existing unsupervised methods and even competitively beating supervised works.

Temporal Event Knowledge Acquisition via Identifying Narratives

Temporal Expression Recognition and Temporal Relationship Extraction from Chinese Narrative Medical Records

Enhancing Narrative Commonsense Reasoning With Multilevel Causal Knowledge

Conditional Generation of Temporally-ordered Event Sequences

Salience-Aware Event Chain Modeling for Narrative Understanding

What Happens Next? Event Prediction Using a Compositional Neural Network Model

Event Causality Is Key to Computational Story Understanding

EventGround: Narrative Reasoning by Grounding to Eventuality-centric Knowledge Graphs

Narrative-of-Thought: Improving Temporal Reasoning of Large Language Models via Recounted Narratives

Whats New? Identifying the Unfolding of New Events in Narratives

A Sequential Model for Classifying Temporal Relations between Intra-Sentence Events

A Narratology-Based Framework for Storyline Extraction

Exploring the Chameleon Effect of Contextual Dynamics in Temporal Knowledge Graph for Event Prediction

Inference of Fine-Grained Event Causality from Blogs and Films

NECE: Narrative Event Chain Extraction Toolkit

Using Context Events in Neural Network Models for Event Temporal Status Identification

A Survey on Event-based News Narrative Extraction

Annotations Are Not All You Need: A Cross-modal Knowledge Transfer Network for Unsupervised Temporal Sentence Grounding.

Extracting Temporal Event Relation with Syntax-guided Graph Transformer

Temporal Knowledge Question Answering via Abstract Reasoning Induction

Interpreting Temporal Knowledge Graph Reasoning (student Abstract)