Abstract:Analysis and detection of complex events in videos require a semantic representation of the video content. Existing video semantic representation methods typically require users to pre-define an exhaustive concept lexicon and manually annotate the presence of the concepts in each video, which is infeasible for real-world video event detection problems. In this paper, we propose an automatic semantic concept discovery scheme by exploiting Internet images and their associated tags. Given a target event and its textual descriptions, we crawl a collection of images and their associated tags by performing text based image search using the noun and verb pairs extracted from the event textual descriptions. The system first identifies the candidate concepts for an event by measuring whether a tag is a meaningful word and visually detectable. Then a concept visual model is built for each candidate concept using a SVM classifier with probabilistic output. Finally, the concept models are applied to generate concept based video representations. We use the TRECVID Multimedia Event Detection (MED) 2013 as our video test set and crawl 400K Flickr images to automatically discover 2, 000 visual concepts. We show significant performance gains of the proposed concept discovery method over different video event detection tasks including supervised event modeling over concept space and semantic based zero-shot retrieval without training examples. Importantly, we show the proposed method of automatic concept discovery outperforms other well-known concept library construction approaches such as Classemes and ImageNet by a large margin (228%) in zero-shot event retrieval. Finally, subjective evaluation by humans also confirms clear superiority of the proposed method in discovering concepts for event representation.

Visual Concept Detection and Real Time Object Detection

An efficient concept detection system via sparse ensemble learning.

A Transformer-Based Object Detector with Coarse-Fine Crossing Representations

Fast Object Retrieval Using Direct Spatial Matching

Universal Object Detection with Large Vision Model

Real-Time Object Detection and Recognition with Computer Vision

Visual-Inertial Object Detection and Mapping

A Trainable System for Object Detection

Image Detection and Real Time Object Detection

A real world detection system

Event-Driven Semantic Concept Discovery by Exploiting Weakly Tagged Internet Images

Automatic Visual Concept Learning for Social Event Understanding

Open-Vocabulary 3D Detection via Image-level Class and Debiased Cross-modal Contrastive Learning

Robust Semantic Concept Detection in Large Video Collections

Virtual Sparse Convolution for Multimodal 3D Object Detection

A Real World Detection System - Combining Color, Shape and Appearance to Enable Real-time Road Sign Detection.

Real-time Transformer-based Open-Vocabulary Detection with Efficient Fusion Head

OSR-ViT: A Simple and Modular Framework for Open-Set Object Detection and Discovery

Remote Sensing Target Detection Based on Visual Saliency Guidance and Classifier Fusion

Video Concept Detection Using Support Vector Machines - TRECVID 2007 Evaluations

Pseudo-labels for Supervised Learning on Dynamic Vision Sensor Data, Applied to Object Detection under Ego-motion