Abstract:Automatic image annotation is a key technology in image understanding and pattern recognition, and is becoming increasingly important in order to annotate large-scale images. In the past decade, the nearest neighbor model-based AIA (Automatic image annotation) methods have been proved to be the most successful in all classical models. This model has four major challenges including semantic gap, label-imbalance, wider range labels, and weak-labeling. In this paper, we propose a novel annotation model based on three-pass KNN (k-Nearest Neighbor) to address the aforementioned challenges. The key idea is to identify appropriate neighbors at each pass KNN. In the first pass KNN, we identify the several most relevant categories based on label feature rather than visual feature as traditional models. In the second pass KNN, we determine the relevant images based on multi-modal (visual and textual label) embedding features. As the test image has not been annotated with any label, we propose a pre-annotation strategory before image annotation to improve the semantic level. In the third pass KNN, we capture relevant labels from semantically and visually similar images and propagate them to the given unlabeled image. In contrast with traditional nearest neighbor based methods, our method can inherently alleviate the problems of semantic gap, label-imbalance, and wider range labels. In addition, to alleviate the issue of weak-labeling, we propose label refinement for training images. Extensive experiments on three classical benchmark datasets and MS-COCO demonstrate that the proposed method significantly outperforms the state-of-the-art in terms of per-label and per-image metrics.

Automatic Image Region Annotation Through Segmentation Based Visual Semantic Analysis and Discriminative Classification

Image Region Annotation Based on Segmentation and Semantic Correlation Analysis

Image Semantic Annotating Method Based on Region Features Relevancy

An Hmm-Svm-Based Automatic Image Annotation Approach

A Novel Automatic Image Annotation Method Based on Co-occurrence Keywords

Labeling images by integrating sparse multiple distance learning and semantic context modeling

A stratification-based approach to accurate and fast image annotation

A Semantic Context Model For Automatic Image Annotation

Automatic image annotation using semantic relevance

Supervised Semantic Image Annotation Using Region Relevance

An interactive semi-supervised approach for automatic image annotation

Image Region Labeling by Exploring Contextual Information of Visual Spatial and Semantic Concepts

Automatic image annotation based on salient regions

The image annotation algorithm using convolutional features from intermediate layer of deep learning

Ensemble of Multiple Descriptors for Automatic Image Annotation

Automatic image annotation via local multi-label classification

Exploiting Spatial Context Constraints for Automatic Image Region Annotation.

Hybrid Generative/Discriminative Learning for Automatic Image Annotation

Image region label refinement using spatial position relation graph.

Automatic Image Annotation by Sequentially Learning From Multi-Level Semantic Neighborhoods

Automatic image annotation by using concept-sensitive salient objects for image content representation.