Image Search by modality analysis: A study of color semantics
Xiaohui Wang,Jia Jia,Jiaming Yin,Yongxin Wang,Lianhong Cai
2011-01-01
Abstract:In this paper, we present a novel image search system, color-based image search by modality analysis. This system enables users to search images with modality words as query, like "romantic", "quiet", etc. Modality here is defined as emotions and conscious subjective aspect of feeling (interests, attitudes, opinions, etc). Modality presents richer semantics than emotions on human perception and understanding of images. We first propose seven modality dimensions to quantitatively analyze modality. A Chinese corpus containing 1200 modality words is constructed, in which each word is annotated with a seven-dimensional modality vector. Then three-color combinations are extracted from images as visual signatures by utilizing K-means clustering algorithm and CIE L*A*B* color space. To bridge the gap between modality (high-level semantic concepts) and three-color combinations (visual signatures), the prediction model for modality vectors is built up based on Kobayashi's Color Image Scale by using the decision tree algorithm. Finally, we establish the image search system. Objective evaluations show the accuracy of the prediction model for modality vectors, while subjective evaluation results indicate that the extracted three-color combinations are in agreement with human perception and search results of our system are consistent with queries.