Attribute-Augmented Semantic Hierarchy: Towards a Unified Framework for Content-Based Image Retrieval

Hanwang Zhang,Zheng-Jun Zha,Yang Yang,Shuicheng Yan,Yue Gao,Tat-Seng Chua
DOI: https://doi.org/10.1145/2637291
2014-01-01
Abstract:This article presents a novel attribute-augmented semantic hierarchy (A2SH) and demonstrates its effectiveness in bridging both the semantic and intention gaps in content-based image retrieval (CBIR). A2SH organizes semantic concepts into multiple semantic levels and augments each concept with a set of related attributes. The attributes are used to describe the multiple facets of the concept and act as the intermediate bridge connecting the concept and low-level visual content. An hierarchical semantic similarity function is learned to characterize the semantic similarities among images for retrieval. To better capture user search intent, a hybrid feedback mechanism is developed, which collects hybrid feedback on attributes and images. This feedback is then used to refine the search results based on A2SH. We use A2SH as a basis to develop a unified content-based image retrieval system. We conduct extensive experiments on a large-scale dataset of over one million Web images. Experimental results show that the proposed A2SH can characterize the semantic affinities among images accurately and can shape user search intent quickly, leading to more accurate search results as compared to state-of-the-art CBIR solutions.
What problem does this paper attempt to address?