Abstract:Automatic video annotation is an important ingredient for semantic-level video browsing, search and navigation. Much attention has been paid to this topic in recent years. These researches have evolved through two paradigms. In the first paradigm, each concept is individually annotated by a pre-trained binary classifier. However, this method ignores the rich information between the video concepts and only achieves limited success. Evolved from the first paradigm, the methods in the second paradigm add an extra step on the top of the first individual classifiers to fuse the multiple detections of the concepts. However, the performance of these methods can be degraded by the error propagation incurred in the first step to the second fusion one. In this article, another paradigm of the video annotation method is proposed to address these problems. It simultaneously annotates the concepts as well as model correlations between them in one step by the proposed Correlative Multilabel (CML) method, which benefits from the compensation of complementary information between different labels. Furthermore, since the video clips are composed by temporally ordered frame sequences, we extend the proposed method to exploit the rich temporal information in the videos. Specifically, a temporal-kernel is incorporated into the CML method based on the discriminative information between Hidden Markov Models (HMMs) that are learned from the videos. We compare the performance between the proposed approach and the state-of-the-art approaches in the first and second paradigms on the widely used TRECVID data set. As to be shown, superior performance of the proposed method is gained.

Correlative Multi-Label Video Annotation.

Correlative Multilabel Video Annotation with Temporal Kernels

A Unifying Multi-Label Temporal Kernel Machine with Its Application to Video Annotation

Correlative multi-label multi-instance image annotation

Dual Enhancement for Multi-Label Learning with Missing Labels

Ensemble Approach Based on Conditional Random Field for Multi-Label Image and Video Annotation

Sequence Multi-Labeling: A Unified Video Annotation Scheme with Spatial and Temporal Context

Online Multi-Label Active Annotation

Refining Video Annotation by Exploiting Pairwise Concurrent Relation.

Ensemble Multi-Instance Multi-Label Learning Approach for Video Annotation Task

Semi-supervised multi-instance multi-label learning for video annotation task.

An Automatic Video Annotation Method Based on Multiple Complementary Classifiers

Collaborative learning for image and video annotation.

Multi-label video classification via coupling attentional multiple instance learning with label relation graph

MULTI-LABEL IMAGE RECOGNITION WITH JOINT CLASS-AWARE MAP DISENTANGLING AND LABEL CORRELATION EMBEDDING

Multi-Modality Transfer Based on Multi-Graph Optimization for Domain Adaptive Video Concept Annotation

Correlation concept-cognitive learning model for multi-label classification

Video Annotation System Based on Categorizing and Keyword Labelling

Enhancing Label Correlation Feedback in Multi-Label Text Classification via Multi-Task Learning

Learning concepts by modeling relationships

Context-aware focal alignment network for micro-video multi-label classification