Abstract:Automatic image annotation is a key technology in image understanding and pattern recognition, and is becoming increasingly important in order to annotate large-scale images. In the past decade, the nearest neighbor model-based AIA (Automatic image annotation) methods have been proved to be the most successful in all classical models. This model has four major challenges including semantic gap, label-imbalance, wider range labels, and weak-labeling. In this paper, we propose a novel annotation model based on three-pass KNN (k-Nearest Neighbor) to address the aforementioned challenges. The key idea is to identify appropriate neighbors at each pass KNN. In the first pass KNN, we identify the several most relevant categories based on label feature rather than visual feature as traditional models. In the second pass KNN, we determine the relevant images based on multi-modal (visual and textual label) embedding features. As the test image has not been annotated with any label, we propose a pre-annotation strategory before image annotation to improve the semantic level. In the third pass KNN, we capture relevant labels from semantically and visually similar images and propagate them to the given unlabeled image. In contrast with traditional nearest neighbor based methods, our method can inherently alleviate the problems of semantic gap, label-imbalance, and wider range labels. In addition, to alleviate the issue of weak-labeling, we propose label refinement for training images. Extensive experiments on three classical benchmark datasets and MS-COCO demonstrate that the proposed method significantly outperforms the state-of-the-art in terms of per-label and per-image metrics.

Image Annotation By Learning Label-Specific Distance Metrics

Learning Multi-Task Local Metrics for Image Annotation

Distance Metric Learning from Uncertain Side Information with Application to Automated Photo Tagging

Automatic Image Annotation by Sequentially Learning From Multi-Level Semantic Neighborhoods

A Semantic Distance Based Nearest Neighbor Method for Image Annotation.

Image Annotation with Nearest Neighbor Based on Semantic Information

K-Nearest Neighbors Relevance Annotation Model for Distance Education

Integrating distance metric learning into label propagation model for multi-label image annotation

Nearest Labelset Using Double Distances for Multi-label Classification

Ranking-oriented Nearest-Neighbor Based Method for Automatic Image Annotation

Distance-Dependent Metric Learning.

Learning Label-Specific Multiple Local Metrics for Multi-Label Classification

Automatic image annotation using adaptive weighted distance in improved K nearest neighbors framework

Learning a metric for class-conditional KNN

Metric Learning-Guided K Nearest Neighbor Multilabel Classifier.

Labeling images by integrating sparse multiple distance learning and semantic context modeling

Large Margin Weighted K-Nearest Neighbors Label Distribution Learning for Classification.

Automatic Image Annotation Combining Svms and Knn Algorithm

Multi-label image annotation based on multi-model

Effective Multi-Modal Multi-Label Learning for Automatic Image Annotation.

A Weighted Topic Model Learned from Local Semantic Space for Automatic Image Annotation