Indexing Heterogeneous Features with Superimages

Qingjun Luo,Shiliang Zhang,Tiejun Huang,Wen Gao,Qi Tian

DOI: https://doi.org/10.1007/s13735-014-0064-x

2014-01-01

International Journal of Multimedia Information Retrieval

Abstract:As an important procedure in image retrieval, off-line indexing focuses on organizing relevant images together and largely decides the efficiency, accuracy, and memory cost of the retrieval system. Because the image contains multi-level visual and semantic clues, the described indexing strategy should be able to reflect such multi-level relevance. However, most of the existing indexing strategies view database images individually and only consider partial relevance, i.e., relevance reflected by either local or global feature. To overcome these issues and design better indexing strategy, we propose to package semantically relevant images into superimages , and then index superimages instead of single images. Superimage effectively packages multiple images into one new unit, and hence significantly decreases the number of images to be indexed. This naturally saves the memory cost and retrieval time. To make the final index file discriminative to both visual and semantic relevances, we extract local descriptors from superimages and index them with inverted file. During online retrieval, we only need to extract local descriptors from queries, but could get semantic-aware retrieval results. This is because during our off-line indexing stage, both the semantically and visually relevant images are organized together by indexing heterogeneous features in superimages. Therefore, our approach is naturally superior to many online retrieval fusion algorithms in terms of retrieval efficiency and memory consumption. Moreover, extensive experiments on multiple retrieval tasks also manifest the promising accuracy of our approach.