Abstract:Most recently the Bag-of-Features (BoF) representation has been well advocated for image search and classification, with two decent phases named sparse coding and max pooling to compensate quantization loss as well as inject spatial layouts. But still, much information has been discarded by quantizing local descriptors with two-dimensional layouts into a one-dimensional BoF histogram. In this paper, we revisit this popular “sparse coding + max pooling” paradigm by “looking around” the local descriptor context towards an optimal BoF. First, we introduce a Weakly supervised Sparse Coding (WSC) to exploit the Classemes-based attribute labeling to refine the descriptor coding procedure. It is achieved by learning an attribute-to-word co-occurrence prior to impose a label inconsistency distortion over the ℓ1 based coding regularizer, such that the descriptor codes can maximally preserve the image semantic similarity. Second, we propose an adaptive feature pooling scheme over “superpixels” rather than over fixed spatial pyramids, named Geometric Consistency Pooling (GCP). As an effect, local descriptors enjoying good geometric consistency are pooled together to ensure a more precise spatial layouts embedding in BoF. Both of our phases are unsupervised, which differ from the existing works in supervised dictionary learning, sparse coding and feature pooling. Therefore, our approach enables potential applications like scalable visual search. We evaluate in both image classification and search benchmarks and report good improvements over the state-of-the-arts.

How to use Bag-of-Words model better for image classification

Object Recognition Based on the Region of Interest and Optimal Bag of Words Model.

The Extended Bag of Words Model for Visual Recognition and Categorization

Contextual Pooling in Image Classification

Semantics-Preserving Bag-of-Words Models and Applications

Adaptive Aggregating Multiresolution Feature Coding for Image Classification

Considering The Spatial Layout Information Of Bag Of Features (Bof) Framework For Image Classification

Discriminative Image Representation for Classification

Spatial pooling of heterogeneous features for image classification

Improving Bag-of-words Scheme for Scene Categorization

An image classification method based on bag of words model

Hierarchical feature coding for image classification

Robust Feature Encoding with Neighborhood Information for Image Classification

Image Classification Using Spatial Pyramid Coding And Visual Word Reweighting

Image Representation Based on Multiple Visual Codebooks

Spatial Encoding of Visual Words for Image Classification.

Visual Feature Coding for Image Classification Integrating Dictionary Structure

Multi-layer orthogonal visual codebook for image classification

Local Deep Descriptors in Bag-of-Words for Image Retrieval.

Weakly supervised sparse coding with geometric consistency pooling

Image Classification by Exploiting the Spatial Context Information