Abstract:Learning-based video annotation is essential for video analysis and understanding, and many various approaches have been proposed to avoid the intensive labor costs of purely manual annotation. However, there lacks a generic framework due to several difficulties, such as dependence of domain knowledge, insufficiency of training data, no precise localization and inefficacy for large-scale video dataset. In this paper, we propose a novel approach based on semi-supervised learning by means of information from the Internet for interesting event annotation in videos. Concretely, a Fast Graph-based Semi-Supervised Multiple Instance Learning (FGSSMIL) algorithm, which aims to simultaneously tackle these difficulties in a generic framework for various video domains (e. g., sports, news, and movies), is proposed to jointly explore small-scale expert labeled videos and large-scale unlabeled videos to train the models. The expert labeled videos are obtained from the analysis and alignment of well-structured video related text (e. g., movie scripts, web-casting text, close caption). The unlabeled data are obtained by querying related events from the video search engine (e. g., YouTube, Google) in order to give more distributive information for event modeling. Two critical issues of FGSSMIL are: 1) how to calculate the weight assignment for a graph construction, where the weight of an edge specifies the similarity between two data points. To tackle this problem, we propose a novel Multiple Instance Learning Induced Similarity (MILIS) measure by learning instance sensitive classifiers; 2) how to solve the algorithm efficiently for large-scale dataset through an optimization approach. To address this issue, Concave-Convex Procedure (CCCP) and nonnegative multiplicative updating rule are adopted. We perform the extensive experiments in three popular video domains: movies, sports, and news. The results compared with the state-of-the-arts are promising and demonstrate the effectiveness and efficiency of our proposed approach.

Semi-supervised Early Event Detection.

Efficient Early Event Detector for Streaming Sequence.

Early Expression Detection Via Online Multi-Instance Learning with Nonlinear Extension

Reliable Shot Identification for Complex Event Detection Via Visual-Semantic Embedding.

Graph Regularized Structured Output SVM for Early Expression Detection with Online Extension

A generic framework for event detection in various video domains.

A System Based On Sequence Learning For Event Detection In Surveillance Video

Interactive Surveillance Event Detection through Mid-level Discriminative Representation

Embedding and Predicting the Event at Early Stage

Online Kernel-Based Structured Output SVM for Early Expression Detection

Adversarial Training for Weakly Supervised Event Detection

OntoED: Low-resource Event Detection with Ontology Embedding

Event-driven Weakly Supervised Video Anomaly Detection

PKU-NEC @TRECVID2012 SED - Uneven-Sequence Based Event Detection in Surveillance Video.

SD2Event: Self-supervised Learning of Dynamic Detectors and Contextual Descriptors for Event Cameras

Unusual event detection and prediction based on sectional contextual edit distance

Hierarchical Latent Concept Discovery for Video Event Detection

Esur: A System for Events Detection in Surveillance Video

S2L: SINGLE-STREAMLINE FOR COMPLEX VIDEO EVENT DETECTION

A Generic Framework for Video Annotation Via Semi-Supervised Learning.

Soccer Video Event Detection By Fusing Middle Level Visual Semantics Of An Event Clip