Abstract:We often remember images and videos that we have seen or recorded before but cannot quite recall the exact venues or details of the contents. We typically have vague memories of the contents, which can often be expressed as a textual description and/or rough visual descriptions of the scenes. Using these vague memories, we then want to search for the corresponding videos of interest. We call this “Memory Recall based Video Search” (MRVS). To tackle this problem, we propose a video search system that permits a user to input his/her vague and incomplete query as a combination of text query, a sequence of visual queries, and/or concept queries. Here, a visual query is often in the form of a visual sketch depicting the outline of scenes within the desired video, while each corresponding concept query depicts a list of visual concepts that appears in that scene. As the query specified by users is generally approximate or incomplete, we need to develop techniques to handle this inexact and incomplete specification by also leveraging on user feedback to refine the specification. We utilize several innovative approaches to enhance the automatic search. First, we employ a visual query suggestion model to automatically suggest potential visual features to users as better queries. Second, we utilize a color similarity matrix to help compensate for inexact color specification in visual queries. Third, we leverage on the ordering of visual queries and/or concept queries to rerank the results by using a greedy algorithm. Moreover, as the query is inexact and there is likely to be only one or few possible answers, we incorporate an interactive feedback loop to permit the users to label related samples which are visually similar or semantically close to the relevant sample. Based on the labeled samples, we then propose optimization algorithms to update visual queries and concept weights to refine the search results. We conduct experiments on two large-scale video datasets: TRECVID 2010 and YouTube. The experimental results demonstrate that our proposed system is effective for MRVS tasks.

Content-based Video Retrieval Integrating Human Perception.

Unsupervised Teacher-Student Model for Large-Scale Video Retrieval.

Content-based Video Similarity Model

RELEVANCE FEEDBACK OF VIDEO RETRIEVAL

Content-Based Video Retrieval By Example Clip On Www

A New Approach to Retrieve Video by Example Video Clip

Video retrieval with multi-modal features.

Content-Based Video Search: Is there a Need, and Is it Possible?

Video Similarity Measurement

News Video Retrieval By Learning Multimodal Semantic Information

Content-Based Video Retrieval Using Audio and Visual Clues

Multimedia Content-Based Visual Retrieval

A Hybrid Approach to Video Retrieval in a Generic Video Management and Application Processing Framework

Memory recall based video search: Finding videos you have seen before based on your memory

Accommodating hybrid retrieval in a comprehensive video database management system

Semantic-based surveillance video retrieval

Content-Based Image And Video Indexing And Retrieval

An Improved System for Concept-Based Video Retrieval

An integrated semantic-based approach in concept based video retrieval

A Video Content Retrieval Method Based on Spatio-temporal Change Information

A Tree-Based Paradigm for Content-Based Video Retrieval and Management