Abstract:Although it has been studied for years by the computer vision and machine learning communities, image annotation is still far from practical. In this paper, we propose a novel attempt at model-free image annotation, which is a data-driven approach that annotates images by mining their search results. Some 2.4 million images with their surrounding text are collected from a few photo forums to support this approach. The entire process is formulated in a divide-and-conquer framework where a query keyword is provided along with the uncaptioned image to improve both the effectiveness and efficiency. This is helpful when the collected data set is not dense everywhere. In this sense, our approach contains three steps: 1) the search process to discover visually and semantically similar search results, 2) the mining process to identify salient terms from textual descriptions of the search results, and 3) the annotation rejection process to filter out noisy terms yielded by Step 2. To ensure real-time annotation, two key techniques are leveraged-one is to map the high-dimensional image visual features into hash codes, the other is to implement it as a distributed system, of which the search and mining processes are provided as Web services. As a typical result, the entire process finishes in less than 1 second. Since no training data set is required, our approach enables annotating with unlimited vocabulary and is highly scalable and robust to outliers. Experimental results on both real Web images and a benchmark image data set show the effectiveness and efficiency of the proposed algorithm. It is also worth noting that, although the entire approach is illustrated within the divide-and-conquer framework, a query keyword is not crucial to our current implementation. We provide experimental results to prove this.

A Survey of Semantic Image Annotation Tools Towards Digital Humanities

Research On Methodology Of Image Semantic Understanding

Data Modeling and Evaluation of Deep Semantic Annotation for Cultural Heritage Images

Annotating Historical Archives of Images

Development and Application of the Semantic Annotation Framework for Digital Images.

Automatic Image Annotations by Mining Web Image Data

Image Semantic Description and Automatic Semantic Annotation

Machine Annotation and Retrieval for Digital Imagery of Historical Materials

Survey of Keyword Based Image Annotation

Third International IEEE Conference on Signal-Image Technologies and Internet-Based System, SITIS 2007, Shanghai, China, December 16-18, 2007

Image Semantic Annotating Method Based on Region Features Relevancy

A web-based approach for image annotation

Modeling Image Data for Effective Indexing and Retrieval in Large General Image Databases.

An Image Retrieval And Semi-Automatic Annotation Scheme For Large Image Databases On The Web

A Survey and Analysis on Automatic Image Annotation

Automatic semantic annotation of images based on Web data.

Semantic Classification and Annotation of Images

Image annotation: Then and now

Annotating Images by Mining Image Search Results

Image annotation using search and mining technologies.

AnnoSearch: Image Auto-Annotation by Search