Abstract:Although it has been studied for years by the computer vision and machine learning communities, image annotation is still far from practical. In this paper, we propose a novel attempt at model-free image annotation, which is a data-driven approach that annotates images by mining their search results. Some 2.4 million images with their surrounding text are collected from a few photo forums to support this approach. The entire process is formulated in a divide-and-conquer framework where a query keyword is provided along with the uncaptioned image to improve both the effectiveness and efficiency. This is helpful when the collected data set is not dense everywhere. In this sense, our approach contains three steps: 1) the search process to discover visually and semantically similar search results, 2) the mining process to identify salient terms from textual descriptions of the search results, and 3) the annotation rejection process to filter out noisy terms yielded by Step 2. To ensure real-time annotation, two key techniques are leveraged-one is to map the high-dimensional image visual features into hash codes, the other is to implement it as a distributed system, of which the search and mining processes are provided as Web services. As a typical result, the entire process finishes in less than 1 second. Since no training data set is required, our approach enables annotating with unlimited vocabulary and is highly scalable and robust to outliers. Experimental results on both real Web images and a benchmark image data set show the effectiveness and efficiency of the proposed algorithm. It is also worth noting that, although the entire approach is illustrated within the divide-and-conquer framework, a query keyword is not crucial to our current implementation. We provide experimental results to prove this.

Bipartite graph reinforcement model for web image annotation.

An Adaptive Graph Model for Automatic Image Annotation.

A Broadcast Model for Web Image Annotation

A Search-Based Web Image Annotation Method

Representative views re-ranking for 3D model retrieval with multi-bipartite graph reinforcement model.

Annotation-retrieval reinforcement by visual cognition modeling on manifold.

Web and personal image annotation by mining label correlation with relaxed visual graph embedding.

Multi-progressive model for web image annotation.

An Algorithm for the Automatic Annotation Refinement on Large-Scale Web Images

Web Image Annotation Based on Tri-relational Graph and Semantic Context Analysis.

Image annotation by modeling Supporting Region Graph

Multi-Graph Similarity Reinforcement For Image Annotation Refinement

Image Annotation by Large-Scale Content-Based Image Retrieval

Automatic Image Annotations by Mining Web Image Data

Annotating personal albums via web mining.

A web-based approach for image annotation

Improve Image Annotation by Combining Multiple Models

Web video topic discovery and tracking via bipartite graph reinforcement model.

Bridging the Semantic Gap Between Image Contents and Tags

Improving Web Image Retrieval Using Image Annotations and Inference Network

Annotating Images by Mining Image Search Results