Abstract:Analysis and detection of complex events in videos require a semantic representation of the video content. Existing video semantic representation methods typically require users to pre-define an exhaustive concept lexicon and manually annotate the presence of the concepts in each video, which is infeasible for real-world video event detection problems. In this paper, we propose an automatic semantic concept discovery scheme by exploiting Internet images and their associated tags. Given a target event and its textual descriptions, we crawl a collection of images and their associated tags by performing text based image search using the noun and verb pairs extracted from the event textual descriptions. The system first identifies the candidate concepts for an event by measuring whether a tag is a meaningful word and visually detectable. Then a concept visual model is built for each candidate concept using a SVM classifier with probabilistic output. Finally, the concept models are applied to generate concept based video representations. We use the TRECVID Multimedia Event Detection (MED) 2013 as our video test set and crawl 400K Flickr images to automatically discover 2, 000 visual concepts. We show significant performance gains of the proposed concept discovery method over different video event detection tasks including supervised event modeling over concept space and semantic based zero-shot retrieval without training examples. Importantly, we show the proposed method of automatic concept discovery outperforms other well-known concept library construction approaches such as Classemes and ImageNet by a large margin (228%) in zero-shot event retrieval. Finally, subjective evaluation by humans also confirms clear superiority of the proposed method in discovering concepts for event representation.

Context-based visual concept detection using domain adaptive semantic diffusion

Concept detection: Convergence to local features and opportunities beyond

Semantic Context Transfer Across Heterogeneous Sources for Domain Adaptive Video Search.

Domain Adaptive Semantic Diffusion for Large Scale Context-Based Video Annotation

Fast Semantic Diffusion for Large-Scale Context-Based Image and Video Annotation

Exploring Inter-Concept Relationship with Context Space for Semantic Video Indexing

Event-Driven Semantic Concept Discovery by Exploiting Weakly Tagged Internet Images

Video Semantic Concept Detection Using Ontology

Semantic Video Search by Exploiting Large-Scale Visual Concepts

TRECVid 2013 Semantic Video Concept Detection by NTT-MD-DUT.

Context-Dependent Diffusion Network For Visual Relationship Detection

VIREO-374 : LSCOM Semantic Concept Detectors Using Local Keypoint Features

Contextualized Diffusion Models for Text-Guided Image and Video Generation

Video Semantic Concept Detection Based on Conceptual Correlation and Boosting

Video Concept Detection Based on Multiple Features and Classifiers Fusion

Robust Semantic Concept Detection in Large Video Collections

High-Level Video Semantic Concept Detection Based on Multi-level Feature Representations.

A Knowledge-Assisted Framework for Video Semantic Concept Detection

Fusing concept detection and geo context for visual search

Context-Aware Domain Adaptation in Semantic Segmentation

Cross-domain Video Concept Detection: A Joint Discriminative and Generative Active Learning Approach