Abstract:Event-specific concepts are the semantic concepts specifically designed for the events of interest, which can be used as a mid-level representation of complex events in videos. Existing methods only focus on defining event-specific concepts for a small number of pre-defined events, but cannot handle novel unseen events. This motivates us to build a large scale event-specific concept library that covers as many real-world events and their concepts as possible. Specifically, we choose WikiHow, an online forum containing a large number of how-to articles on human daily life events. We perform a coarse-to-fine event discovery process and discover 500 events from WikiHow articles. Then we use each event name as query to search YouTube and discover event-specific concepts from the tags of returned videos. After an automatic filter process, we end up with 95,321 videos and 4,490 concepts. We train a Convolutional Neural Network (CNN) model on the 95,321 videos over the 500 events, and use the model to extract deep learning feature from video content. With the learned deep learning feature, we train 4,490 binary SVM classifiers as the event-specific concept library. The concepts and events are further organized in a hierarchical structure defined by WikiHow, and the resultant concept library is called EventNet. Finally, the EventNet concept library is used to generate concept based representation of event videos. To the best of our knowledge, EventNet represents the first video event ontology that organizes events and their concepts into a semantic structure. It offers great potential for event retrieval and browsing. Extensive experiments over the zero-shot event retrieval task when no training samples are available show that the proposed EventNet concept library consistently and significantly outperforms the state-of-the-art (such as the 20K ImageNet concepts trained with CNN) by a large margin up to 207%. We will also show that EventNet structure can help users find relevant concepts for novel event queries that cannot be well handled by conventional text based semantic analysis alone. The unique two-step approach of first applying event detection models followed by detection of event-specific concepts also provides great potential to improve the efficiency and accuracy of Event Recounting since only a very small number of event-specific concept classifiers need to be fired after event detection.

Event oriented dictionary learning for complex event detection

EventNet: A Large Scale Structured Concept Library for Complex Event Detection in Video

Event-Driven Semantic Concept Discovery by Exploiting Weakly Tagged Internet Images

Enhancing Video Event Recognition Using Automatically Constructed Semantic-Visual Knowledge Base.

Abnormal event detection via multi-instance dictionary learning

How Unlabeled Web Videos Help Complex Event Detection?

Semantic Concept Discovery for Large-Scale Zero-Shot Event Detection.

Zero-Shot Video Event Detection With High-Order Semantic Concept Discovery and Matching

Reliable Shot Identification for Complex Event Detection Via Visual-Semantic Embedding.

Building A Large Concept Bank for Representing Events in Video

Web Video Event Recognition by Semantic Analysis from Ubiquitous Documents

Columbia-UCF TRECVID2010 Multimedia Event Detection: Combining Multiple Modalities, Contextual Concepts, and Temporal Matching.

Hierarchical Latent Concept Discovery for Video Event Detection

Dynamic Concept Composition for Zero-Example Event Detection

OntoED: Low-resource Event Detection with Ontology Embedding

Deep Fusion of Multiple Semantic Cues for Complex Event Recognition

They Are Not Equally Reliable: Semantic Event Search Using Differentiated Concept Classifiers

High-level Event Recognition in Unconstrained Videos

SD2Event: Self-supervised Learning of Dynamic Detectors and Contextual Descriptors for Event Cameras

Large-scale Video Event Classification Using Dynamic Temporal Pyramid Matching of Visual Semantics

Complex Event Detection Via Multi-source Video Attributes.