Abstract:Given a textual query in traditional text-based image retrieval (TBIR), relevant images are to be reranked using visual features after the initial text-based search. In this paper, we propose a new bag-based reranking framework for large-scale TBIR. Specifically, we first cluster relevant images using both textual and visual features. By treating each cluster as a "bag" and the images in the bag as "instances," we formulate this problem as a multi-instance (MI) learning problem. MI learning methods such as mi-SVM can be readily incorporated into our bag-based reranking framework. Observing that at least a certain portion of a positive bag is of positive instances while a negative bag might also contain positive instances, we further use a more suitable generalized MI (GMI) setting for this application. To address the ambiguities on the instance labels in the positive and negative bags under this GMI setting, we develop a new method referred to as GMI-SVM to enhance retrieval performance by propagating the labels from the bag level to the instance level. To acquire bag annotations for (G)MI learning, we propose a bag ranking method to rank all the bags according to the defined bag ranking score. The top ranked bags are used as pseudopositive training bags, while pseudonegative training bags can be obtained by randomly sampling a few irrelevant images that are not associated with the textual query. Comprehensive experiments on the challenging real-world data set NUS-WIDE demonstrate our framework with automatic bag annotation can achieve the best performances compared with existing image reranking methods. Our experiments also demonstrate that GMI-SVM can achieve better performances when using the manually labeled training bags obtained from relevance feedback.

Generalized Manifold-Ranking-Based Image Retrieval

Learning an Image Manifold for Retrieval

Learning with Both Unlabeled Data and Query Logs for Image Search.

Efficient Manifold Ranking for Image Retrieval

Integrated Manifold Ranking and Region Matching for Image Retrieval

Multistage and Multi-features Medical Image Retrieval System

Multi-Manifold Ranking: Using Multiple Features for Better Image Retrieval.

Multi-Query Parallel Field Ranking for image retrieval.

Learning In Region-Based Image Retrieval

EMR: A Scalable Graph-Based Ranking Model for Content-Based Image Retrieval

Regularized Regression on Image Manifold for Retrieval

Manifold-Ranking-Based Keyword Propagation for Image Retrieval

Improving Web image search by bag-based reranking.

Hypergraph-based multi-example ranking with sparse representation for transductive learning image retrieval

Multilabel Neighborhood Propagation for Region-Based Image Retrieval

Multiple random walk and its application in content-based image retrieval.

Correlated Probabilistic Label Propagation for Region-Based Image Retrieval

A comprehensive study on learning to rank for content-based image retrieval

Image-based 3D Model Retrieval Using Manifold Learning

Content Based Image Retrieval Using Manifold-Ranking of Blocks.

Learning to Combine Ad-hoc Ranking Functions for Image Retrieval