Abstract:Automatic image annotation is a key technology in image understanding and pattern recognition, and is becoming increasingly important in order to annotate large-scale images. In the past decade, the nearest neighbor model-based AIA (Automatic image annotation) methods have been proved to be the most successful in all classical models. This model has four major challenges including semantic gap, label-imbalance, wider range labels, and weak-labeling. In this paper, we propose a novel annotation model based on three-pass KNN (k-Nearest Neighbor) to address the aforementioned challenges. The key idea is to identify appropriate neighbors at each pass KNN. In the first pass KNN, we identify the several most relevant categories based on label feature rather than visual feature as traditional models. In the second pass KNN, we determine the relevant images based on multi-modal (visual and textual label) embedding features. As the test image has not been annotated with any label, we propose a pre-annotation strategory before image annotation to improve the semantic level. In the third pass KNN, we capture relevant labels from semantically and visually similar images and propagate them to the given unlabeled image. In contrast with traditional nearest neighbor based methods, our method can inherently alleviate the problems of semantic gap, label-imbalance, and wider range labels. In addition, to alleviate the issue of weak-labeling, we propose label refinement for training images. Extensive experiments on three classical benchmark datasets and MS-COCO demonstrate that the proposed method significantly outperforms the state-of-the-art in terms of per-label and per-image metrics.

Automatic Region-Based Image Annotation Using An Improved Multiple-Instance Learning Algorithm

Region-Based Image Annotation Using Heuristic Support Vector Machine in Multiple-Instance Learning

Automatic image annotation by an iterative approach: incorporating keyword correlations and region matching

Automatic Image Annotation by Sequentially Learning From Multi-Level Semantic Neighborhoods

A Novel Automatic Image Annotation Method Based on Co-occurrence Keywords

Multiple Instance Learning for Automatic Image Annotation.

Transductive Multi-Instance Multi-Label Learning Algorithm with Application to Automatic Image Annotation

Automatic image annotation using adaptive weighted distance in improved K nearest neighbors framework

Automatic Image Annotation Combining Svms and Knn Algorithm

An Empirical Study of Automatic Image Annotation Through Multi-Instance Multi-Label Learning

Automatic image annotation based on the multiple-instance learning

MR-MIL: Manifold ranking based multiple-instance learning for automatic image annotation

A Multi-feature Fusion Method for Automatic Multi-label Image Annotation with Weighted Histogram Integral and Closure Regions Counting

The image annotation algorithm using convolutional features from intermediate layer of deep learning

Automatic tag-to-region assignment via multiple instance learning

Automatic Image Annotation Based on Multi-Auxiliary Information

Semi-Supervised Learning Model Based Efficient Image Annotation

Image Annotation By Multiple-Instance Learning With Discriminative Feature Mapping and Selection

MRS-MIL: Minimum Reference Set Based Multiple Instance Learning for Automatic Image Annotation.

Automatic image annotation based on salient regions

Multiple Kernel Learning Based on Weak Learner for Automatic Image Annotation.