Abstract:The Bag-of-Words (BoW) model is a promising image representation technique for image categorization and annotation tasks. One critical limitation of existing BoW models is that much semantic information is lost during the codebook generation process, an important step of BoW. This is because the codebook generated by BoW is often obtained via building the codebook simply by clustering visual features in Euclidian space. However, visual features related to the same semantics may not distribute in clusters in the Euclidian space, which is primarily due to the semantic gap between low-level features and high-level semantics. In this paper, we propose a novel scheme to learn optimized BoW models, which aims to map semantically related features to the same visual words. In particular, we consider the distance between semantically identical features as a measurement of the semantic gap, and attempt to learn an optimized codebook by minimizing this gap, aiming to achieve the minimal loss of the semantics. We refer to such kind of novel codebook as semantics-preserving codebook (SPC) and the corresponding model as the Semantics-Preserving Bag-of-Words (SPBoW) model. Extensive experiments on image annotation and object detection tasks with public testbeds from MIT's Labelme and PASCAL VOC challenge databases show that the proposed SPC learning scheme is effective for optimizing the codebook generation process, and the SPBoW model is able to greatly enhance the performance of the existing BoW model.

A More Effective Method for Image Representation: Topic Model Based on Latent Dirichlet Allocation.

Topic Discovery Based on LDA_col Model and Topic Significance Re-ranking.

Probabilistic Latent Semantic Analysis for Sketch-Based 3D Model Retrieval

Mining Coherent Topics in Documents Using Word Embeddings and Large-Scale Text Data

Incorporating Knowledge Graph Embeddings into Topic Modeling

Topic Level Sampling Towards Optimized Locality Sensitive Vocabulary Coding

A Feature-Word-topic Model for Image Annotation and Retrieval

Topic models for image annotation and text illustration

Bag-of-Discriminative-Words (BoDW) Representation via Topic Modeling.

Efficient Probabilistic Latent Semantic Analysis with Sparsity Control

Document Clustering Based on Probabilistic Topic Model

Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings

Topic-weak-correlated Latent Dirichlet Allocation

Building Pair-Wise Visual Word Tree For Efficent Image Re-Ranking

Semantics-Preserving Bag-of-Words Models and Applications

Topic2Vec: Learning distributed representations of topics

E2BoWs: an End-to-end Bag-of-Words Model Via Deep Convolutional Neural Network for Image Retrieval

A Joint Model Of Extended Lda And Ibtm Over Streaming Chinese Short Texts

A New Approach to Speeding Up Topic Modeling

Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey

Learning attribute-aware dictionary for image classification and search