Abstract:Automatic image annotation is a key technology in image understanding and pattern recognition, and is becoming increasingly important in order to annotate large-scale images. In the past decade, the nearest neighbor model-based AIA (Automatic image annotation) methods have been proved to be the most successful in all classical models. This model has four major challenges including semantic gap, label-imbalance, wider range labels, and weak-labeling. In this paper, we propose a novel annotation model based on three-pass KNN (k-Nearest Neighbor) to address the aforementioned challenges. The key idea is to identify appropriate neighbors at each pass KNN. In the first pass KNN, we identify the several most relevant categories based on label feature rather than visual feature as traditional models. In the second pass KNN, we determine the relevant images based on multi-modal (visual and textual label) embedding features. As the test image has not been annotated with any label, we propose a pre-annotation strategory before image annotation to improve the semantic level. In the third pass KNN, we capture relevant labels from semantically and visually similar images and propagate them to the given unlabeled image. In contrast with traditional nearest neighbor based methods, our method can inherently alleviate the problems of semantic gap, label-imbalance, and wider range labels. In addition, to alleviate the issue of weak-labeling, we propose label refinement for training images. Extensive experiments on three classical benchmark datasets and MS-COCO demonstrate that the proposed method significantly outperforms the state-of-the-art in terms of per-label and per-image metrics.

Image Annotation Based on Positive-Negative Instances Learning

An efficient and effective automatic image annotation using positive and negative example images

The image annotation algorithm using convolutional features from intermediate layer of deep learning

Multiple Instance Learning for Automatic Image Annotation.

The Image Annotation Method by Convolutional Features from Intermediate Layer of Deep Learning Based on Internet of Things

Automatic image annotation based on the multiple-instance learning

Image annotation using the summation of negative probability based on SVM

A Novel Automatic Image Annotation Method Based on Co-occurrence Keywords

Automatic Image Annotation by Sequentially Learning From Multi-Level Semantic Neighborhoods

Utilization of Rotation-Invariant Uniform LBP Histogram Distribution and Statistics of Connected Regions in Automatic Image Annotation Based on Multi-Label Learning.

Semi-automatic Dynamic Auxiliary-Tag-aided Image Annotation

A web-based approach for image annotation

Automatic Image Annotation with Weakly Labeled Dataset

Multilabel image annotation based on double-layer PLSA model.

Transductive Multi-Instance Multi-Label Learning Algorithm with Application to Automatic Image Annotation

An Empirical Study of Automatic Image Annotation Through Multi-Instance Multi-Label Learning

A Survey on Automatic Image Annotation.

Semi-Supervised Learning Model Based Efficient Image Annotation

Log-based collaborative automatic image annotation

Automatic image annotation with cooperation of concept-specific and universal visual vocabularies

Hierarchical Image Automatic Annotation Based on Discriminative and Generative Models