Abstract:Multimedia event detection aims to precisely retrieve videos that contain complex semantic events from a large pool. This work addresses this task under a zero-shot setting, where only brief event-specific textural information (such as event names, a few descriptive sentences, etc.) is known yet none positive video example is provided. Mainstream approaches to tackling this task are middle-level semantic concept-based, where meticulously-crafted concept banks (e.g., LSCOM) are adopted. We argue that these concept banks are still inadequate facing video semantic complexity. Existing semantic concepts are essentially first-order, mainly designed for atomic objects, scenes or human actions, etc. This work advocates the utilization of high-order concepts (such as subject-predicate-object triplets or adjective-object). The main contributions are two-fold. First, we harvest a comprehensive albeit compact high-order concept library through distilling information from three large public datasets (MS-COCO, Visual Genome, and Kinetics-600), mainly related to visual relations and human-object interactions. Secondly, zero-shot events are often only briefly and partially described via textual input. The resultant semantic ambiguity makes the pursuit of the most indicative high-order concepts challenging. We thus design a novel query-expanding scheme that enriches ambiguous event-specific keywords by searching over either large common knowledge bases (e.g., WikiHow) or top-ranked webpages retrieved from modern search engines. This way sets up a more faithful connection between zero-shot events and high-order concepts. To our best knowledge, this is the first work that strives for concept-based video search beyond first-order concepts. Extensive experiments have been conducted on several large video benchmarks (TRECVID 2013, TRECVID 2014, and ActivityNet-1.3). The evaluations clearly demonstrate the superiority of our constructed high-order concept library and it- complementariness to existing concepts.

Complex Video Event Detection Via Pairwise Fusion of Trajectory and Multi-Label Hypergraphs

Complex Event Detection Via Multi-source Video Attributes.

Integrating Graph Partitioning and Matching for Trajectory Analysis in Video Surveillance

Multiple Hypergraph Ranking for Video Concept Detection

Deep Fusion of Multiple Semantic Cues for Complex Event Recognition

Integrated Multi-Scale Event Verification In An Augmented Foreground Motion Space

A new hybrid information fusion method for trajectory prediction

A Hybrid Graph Network for Complex Activity Detection in Video

Multi-object Tracking Via MHT with Multiple Information Fusion in Surveillance Video

Discovering Joint Audio–visual Codewords for Video Event Detection

Video Visual Relation Detection Via Multi-modal Feature Fusion

Unsupervised Multi-Target Trajectory Detection, Learning And Analysis In Complicated Environments

Detecting Rare Actions and Events from Surveillance Big Data with Bag of Dynamic Trajectories

Modality Mixture Projections for Semantic Video Event Detection

Joint Object Detection and Multi-Object Tracking Based on Hypergraph Matching

Multi-object Events Recognition from Video Sequences Using Extended Finite State Machine

Zero-Shot Video Event Detection With High-Order Semantic Concept Discovery and Matching

Discovering Latent Discriminative Patterns for Multi-Mode Event Representation.

Event Detection In Basketball Video Using Multiple Modalities

Fusing Motion Patterns and Key Visual Information for Semantic Event Recognition in Basketball Videos

Hypergraph-Based Multi-View Action Recognition Using Event Cameras