Abstract:Automatic video annotation is an important ingredient for semantic-level video browsing, search and navigation. Much attention has been paid to this topic in recent years. These researches have evolved through two paradigms. In the first paradigm, each concept is individually annotated by a pre-trained binary classifier. However, this method ignores the rich information between the video concepts and only achieves limited success. Evolved from the first paradigm, the methods in the second paradigm add an extra step on the top of the first individual classifiers to fuse the multiple detections of the concepts. However, the performance of these methods can be degraded by the error propagation incurred in the first step to the second fusion one. In this article, another paradigm of the video annotation method is proposed to address these problems. It simultaneously annotates the concepts as well as model correlations between them in one step by the proposed Correlative Multilabel (CML) method, which benefits from the compensation of complementary information between different labels. Furthermore, since the video clips are composed by temporally ordered frame sequences, we extend the proposed method to exploit the rich temporal information in the videos. Specifically, a temporal-kernel is incorporated into the CML method based on the discriminative information between Hidden Markov Models (HMMs) that are learned from the videos. We compare the performance between the proposed approach and the state-of-the-art approaches in the first and second paradigms on the widely used TRECVID data set. As to be shown, superior performance of the proposed method is gained.

Transductive Video Annotation Via Local Learnable Kernel Classifier

Video Annotation Based on Kernel Linear Neighborhood Propagation

Face Annotation Using Transductive Kernel Fisher Discriminant

Kernel-Based Linear Neighborhood Propagation for Semantic Video Annotation

Transductive Inference With Hierarchical Clustering For Video Annotation

Video Annotation by Graph-Based Learning with Neighborhood Similarity.

Transductive multi-distance learning for video search

Graph-based Transductive Learning for Robust Visual Tracking.

A Unifying Multi-Label Temporal Kernel Machine with Its Application to Video Annotation

Local Attention Sequence Model for Video Object Detection

Semi-supervised Kernel Density Estimation for Video Annotation

Correlative Multilabel Video Annotation with Temporal Kernels

Correlative Linear Neighborhood Propagation for Video Annotation

Collaborative learning for image and video annotation.

Beyond distance measurement: Constructing neighborhood similarity for video annotation

Joint Inductive and Transductive Learning for Video Object Segmentation

Recognizing Key Segments of Videos for Video Annotation by Learning from Web Image Sets

Beyond Distance Measurement: Constructing Neighborhood Similarity for Video Annotation.