Abstract:Existing video search engines have not taken the advantages of video content analysis and semantic understanding. Video search in academia uses semantic annotation to approach content-based indexing. We argue this is a promising direction to enable real content-based video search. However, due to the complexity of both video data and semantic concepts, existing techniques on automatic video annotation are still not able to handle large-scale video set and large-scale concept set, in terms of both annotation accuracy and computation cost. To address this problem, in this paper, we propose a scalable framework for annotation-based video search, as well as a novel approach to enable large-scale semantic concept annotation, that is, online multi-label active learning. This framework is scalable to both the video sample dimension and concept label dimension. Large-scale unlabeled video samples are assumed to arrive consecutively in batches with an initial pre-labeled training set, based on which a preliminary multi-label classifier is built. For each arrived batch, a multi-label active learning engine is applied, which automatically selects and manually annotates a set of unlabeled sample-label pairs. And then an online learner updates the original classifier by taking the newly labeled sample-label pairs into consideration. This process repeats until all data are arrived. During the process, new labels, even without any pre-labeled training samples, can be incorporated into the process anytime. Experiments on TRECVID dataset demonstrate the effectiveness and efficiency of the proposed framework.

Active learning in multimedia annotation and retrieval: A survey

Literature survey of active learning in multimedia annotation and retrieval

Active Video Annotation

Visualization-Based Active Learning for Video Annotation.

Semi-automatic video annotation based on active learning with multiple complementary predictors

Multi-Concept Multi-Modality Active Learning for Interactive Video Annotation

Interactive Video Annotation By Multi-Concept Multi-Modality Active Learning

Semi-Automatic Video Semantic Annotation Based On Active Learning

Semi-supervised SVM Batch Mode Active Learning for Image Retrieval

Semisupervised SVM Batch Mode Active Learning with Applications to Image Retrieval

Survey on Active Learning Algorithms

A comprehensive survey on deep active learning in medical image analysis

MULTI-CLASS SUPPORT VECTOR MACHINE ACTIVE LEARNING FOR MUSIC ANNOTATION

Unlabeled data selection for active learning in image classification

Interactive Video Indexing with Statistical Active Learning

An Interactive Video Annotation Frameowrk with Multiple Modalities

Learning Distinctive Margin Toward Active Domain Adaptation

Online Multi-Label Active Annotation

Active Learning on Sparse Graph for Image Annotation