Abstract:Multimedia event detection aims to precisely retrieve videos that contain complex semantic events from a large pool. This work addresses this task under a zero-shot setting, where only brief event-specific textural information (such as event names, a few descriptive sentences, etc.) is known yet none positive video example is provided. Mainstream approaches to tackling this task are middle-level semantic concept-based, where meticulously-crafted concept banks (e.g., LSCOM) are adopted. We argue that these concept banks are still inadequate facing video semantic complexity. Existing semantic concepts are essentially first-order, mainly designed for atomic objects, scenes or human actions, etc. This work advocates the utilization of high-order concepts (such as subject-predicate-object triplets or adjective-object). The main contributions are two-fold. First, we harvest a comprehensive albeit compact high-order concept library through distilling information from three large public datasets (MS-COCO, Visual Genome, and Kinetics-600), mainly related to visual relations and human-object interactions. Secondly, zero-shot events are often only briefly and partially described via textual input. The resultant semantic ambiguity makes the pursuit of the most indicative high-order concepts challenging. We thus design a novel query-expanding scheme that enriches ambiguous event-specific keywords by searching over either large common knowledge bases (e.g., WikiHow) or top-ranked webpages retrieved from modern search engines. This way sets up a more faithful connection between zero-shot events and high-order concepts. To our best knowledge, this is the first work that strives for concept-based video search beyond first-order concepts. Extensive experiments have been conducted on several large video benchmarks (TRECVID 2013, TRECVID 2014, and ActivityNet-1.3). The evaluations clearly demonstrate the superiority of our constructed high-order concept library and it- complementariness to existing concepts.

An efficient concept detection system via sparse ensemble learning.

SSF: Sparse Point Cloud Object Detection Based on Self-Adaptive Voxel Encoding and Focal-Sparse Convolution

Visual Concept Detection and Real Time Object Detection

Robust Semantic Concept Detection in Large Video Collections

Event-Driven Semantic Concept Discovery by Exploiting Weakly Tagged Internet Images

Representations of Keypoint-Based Semantic Concept Detection: A Comprehensive Study

ConceptLearner: Discovering Visual Concepts from Weakly Labeled Image Collections

Semantic Concept Discovery for Large-Scale Zero-Shot Event Detection.

Parallel Lasso for Large-Scale Video Concept Detection

VIREO-374 : LSCOM Semantic Concept Detectors Using Local Keypoint Features

Boosted Near-miss Under-sampling on SVM ensembles for concept detection in large-scale imbalanced datasets

Exploiting Concept Association to Boost Multimedia Semantic Concept Detection

Integrating knowledge-based sparse representation for image detection

Video Concept Detection Using Support Vector Machines - TRECVID 2007 Evaluations

Discovering Visual Concept Structure with Sparse and Incomplete Tags

Zero-Shot Video Event Detection With High-Order Semantic Concept Discovery and Matching

Automatic Visual Concept Learning for Social Event Understanding

DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detection

Building A Large Concept Bank for Representing Events in Video

Salient Object Detection on Large-Scale Video Data

Semantic Video Search by Exploiting Large-Scale Visual Concepts