Abstract:The idea of representing images using a bag of visual words is currently popular in object category recognition. Since this representation is typically constructed using unsupervised clustering, the resulting visual words may not capture the desired information. Recent work has explored the construction of discriminative visual codebooks that explicitly consider object category information. However, since the codebook generation process is still disconnected from that of classifier training, the set of resulting visual words, while individually discriminative, may not be those best suited for the classifier This paper proposes a novel optimization framework that unifies codebook generation with classifier training. In our approach, each image feature is encoded by a sequence of "visual bits" optimized for each category. An image, which can contain objects from multiple categories, is represented using aggregates of visual bits for each category. Classifiers associated with different categories determine how well a given image corresponds to each category. Based on the performance of these classifiers on the training data, we augment the visual words by generating additional bits. The classifiers are then updated to incorporate the new representation. These two phases are repeated until the desired performance is achieved. Experiments compare our approach to standard clustering-based methods and with state-of-the-art discriminative visual codebook generation. The significant improvements over previous techniques clearly demonstrate the value of unifying representation and classification into a single optimization framework.

Visual Words Ambiguity Analysis in BOW Model

Object Recognition Based on the Region of Interest and Optimal Bag of Words Model.

Visual Words Refining Exploiting Spatial Co-Occurrence Table

Understanding bag-of-words model: a statistical framework

Visual word coding based on difference maximization.

Semantics-Preserving Bag-of-Words Models and Applications

Image classification by visual bag-of-words refinement and reduction

Local Word Bag Model for Text Categorization

Learning Descriptive Visual Representation by Semantic Regularized Matrix Factorization.

Visual language modeling for image classification.

Unifying Discriminative Visual Codebook Generation with Classifier Training for Object Category Recognition

Exploiting visual word co-occurrence for image retrieval.

A Combination of Bag of Categorized Visual Words and Tag Voting Based Image Retrieval

Learning Descriptive Visual Representation for Image Classification and Annotation

BoW-MSN: A Novel Texture Classification Approach.

Visual Word Proximity and Linguistics for Semantic Video Indexing and Near-Duplicate Retrieval

Exploring Spatial Correlation for Visual Object Retrieval

Evaluating Bag-of-visual-words Representations in Scene Classification

Relevance of useful visual words in object retrieval

Scale-invariant visual language modeling for object categorization

Latent Topic Visual Language Model for Object Categorization.