Abstract:Bag-of-visual Words (BoW) image representation has been illustrated as one of the most promising solutions for large-scale near-duplicated image retrieval. However, the traditional visual vocabulary is created in an unsupervised way by clustering a large number of image local features. This is not ideal because it largely ignores the semantic and spatial contexts between local features. In this paper, we propose the geometric visual vocabulary which captures the spatial contexts by quantizing local features in bi-space, i.e., in descriptor space and orientation space. Then, we propose to capture the semantic context by learning a semantic-aware distance metric between local features, which could reasonably measure the semantic similarities between image patches, from which the local features are extracted. The learned distance is hence utilized to cluster the local features for semantic visual vocabulary generation. Finally, we combine the spatial and semantic contexts in a unified framework by extracting local feature groups, computing the spatial configurations between the local features inside the group, and learning a semantic-aware distance between groups. The learned group distance is then utilized to cluster the extracted local feature groups to generate a novel visual vocabulary, i.e., the contextual visual vocabulary. The proposed visual vocabularies, i.e., geometric visual vocabulary, semantic visual vocabulary and contextual visual vocabulary are tested in large-scale near-duplicated image retrieval applications. The geometric visual vocabulary and semantic visual vocabulary achieve better performance than the traditional visual vocabulary. Moreover, the contextual visual vocabulary, which combines both spatial and semantic clues outperforms the state-of-the-art bundled feature in both retrieval precision and efficiency.

ObjectBook Construction for Large-Scale Semantic-Aware Image Retrieval

Fast Object Retrieval Using Direct Spatial Matching

ObjectPatchNet: Towards Scalable and Semantic Image Annotation and Retrieval

Modeling Image Data for Effective Indexing and Retrieval in Large General Image Databases.

PartBook for Image Parsing

Image Annotation by Large-Scale Content-Based Image Retrieval

Real-Time Image Semantic Retrieval Based on VQ

Multi-Scale Visual Words For Object-Based Web Image Search

Efficient Indexing for Large Scale Visual Search

Modeling spatial and semantic cues for large-scale near-duplicated image retrieval

Object-Centric Open-Vocabulary Image-Retrieval with Aggregated Features

Semantic Classification and Annotation of Images

Object-Based Image Retrieval With Attention Analysis And Spatial Re-Ranking

Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasets

Efficient Indexing for Large-Scale Image Search

Fine-Grained Image Search

Weighted Transformable Spatial Pyramid And Scalable Query For Object Retrieval

Query-Adaptive Small Object Search Using Object Proposals and Shape-Aware Descriptors

Image Annotations Based on Semi-supervised Clustering with Semantic Soft Constraints.

ImageSaker : A Semantic-based Image Retrieval System Refining with Concept Model

Superimage: Packing Semantic-Relevant Images for Indexing and Retrieval