A Novel Visual Words Definition Algorithm of Image Patch Based on Contextual Semantic Information

Zhengding Qiu

2010-01-01

Abstract:Visual Words (VWs) representation has recently become popular for scene classification.Visual words are usually constructed by using an unsupervised method to cluster the appearance descriptor vectors of patches.However,the traditional visual words method considers nothing about semantic information.To overcome this defect,this paper proposes the novel contextual semantic constrained Markov Random Fields (MRF) visual words.There are two areas of novelty:first,the useful contextual semantic constrained information are the semantic co-occurrence probabilities of image patches obtained by probabilistic Latent Semantic Analysis (pLSA) instead of by manually labeled.And then we introduce the pseudo-likelihood of labeling to combine the feature appearance similarity and contextual semantic information together in order to provide the more accurate visual words representation.Once these visual words are obtained,the frequency of these visual words in an image forms a histogram which is subsequently used in a scene categorization task by the Support Vector Machine (SVM) classifier.The experimental results compared with the existing methods show that contextual semantic constraints indeed do benefit for visual words selection.