Abstract:Video moment retrieval targets at retrieving a golden moment in a video for a given natural language query. The main challenges of this task include 1) the requirement of accurately localizing (i.e., the start time and the end time of) the relevant moment in an untrimmed video stream, and 2) bridging the semantic gap between textual query and video contents. To tackle those problems, early approaches adopt the sliding window or uniform sampling to collect video clips first and then match each clip with the query to identify relevant clips. Obviously, these strategies are time-consuming and often lead to unsatisfied accuracy in localization due to the unpredictable length of the golden moment. To avoid the limitations, researchers recently attempt to directly predict the relevant moment boundaries without the requirement to generate video clips first. One mainstream approach is to generate a multimodal feature vector for the target query and video frames (e.g., concatenation) and then use a regression approach upon the multimodal feature vector for boundary detection. Although some progress has been achieved by this approach, we argue that those methods have not well captured the cross-modal interactions between the query and video frames. In this paper, we propose an Attentive Cross-modal Relevance Matching (ACRM) model which predicts the temporal boundaries based on an interaction modeling between two modalities. In addition, an attention module is introduced to automatically assign higher weights to query words with richer semantic cues, which are considered to be more important for finding relevant video contents. Another contribution is that we propose an additional predictor to utilize the internal frames in the model training to improve the localization accuracy. Extensive experiments on two public datasets TACoS and Charades-STA demonstrate the superiority of our method over several state-of-the-art methods. Ablation studies have been also conducted to exami-e the effectiveness of different modules in our ACRM model.

Semantic Video Moment Retrieval by Temporal Feature Perturbation and Refinement

Partial Annotation-based Video Moment Retrieval Via Iterative Learning

Video Moment Retrieval with Noisy Labels

QD-VMR: Query Debiasing with Contextual Understanding Enhancement for Video Moment Retrieval

Not All Inputs Are Valid: Towards Open-Set Video Moment Retrieval Using Language

Semantic Modulation Based Residual Network for Temporal Language Queries Grounding in Video.

Modal-specific Pseudo Query Generation for Video Corpus Moment Retrieval

Weakly-Supervised Video Moment Retrieval Via Semantic Completion Network

Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language

Filling the Information Gap Between Video and Query for Language-Driven Moment Retrieval

Frame-Wise Cross-Modal Matching for Video Moment Retrieval

Towards Generalisable Video Moment Retrieval: Visual-Dynamic Injection to Image-Text Pre-Training

Towards Balanced Alignment: Modal-Enhanced Semantic Modeling for Video Moment Retrieval

MLLM as Video Narrator: Mitigating Modality Imbalance in Video Moment Retrieval

Deconfounded Video Moment Retrieval with Causal Intervention

Video Moment Retrieval from Text Queries via Single Frame Annotation

Prompt-based Zero-shot Video Moment Retrieval

Hybrid-Learning Video Moment Retrieval across Multi-Domain Labels

Attentive Moment Retrieval in Videos

Event-aware Video Corpus Moment Retrieval

VLANet: Video-Language Alignment Network for Weakly-Supervised Video Moment Retrieval