Visual feature coding for image classification integrating dictionary structure

YangYu-Bin,ZhuQi-Hai,MaoXiao-Jiao,PanLin-Yan

IF: 8

2015-01-01

Pattern Recognition

Abstract:Methods based on the Bag-Of-Words (BoW) model have made a remarkable success in image classification, but many of which do not consider the structure underlying in the dictionary itself in the feat...

What problem does this paper attempt to address?

Visual Feature Coding for Image Classification Integrating Dictionary Structure

Yu-Bin Yang,Qi-Hai Zhu,Xiao-Jiao Mao,Lin-Yan Pan

DOI: https://doi.org/10.1016/j.patcog.2015.03.012

IF: 8

2015-01-01

Pattern Recognition

Abstract:Methods based on the Bag-Of-Words (BoW) model have made a remarkable success in image classification, but many of which do not consider the structure underlying in the dictionary itself in the feature coding procedure. In this paper, we propose a novel visual feature coding strategy by integrating dictionary structure, into which it incorporates not only the relations between visual features and visual words, but the pair-wise relations among visual words. We further develop an approximating solution to the integrated coding procedure by finding the k-dense neighbors (kDN) of a feature, rather than its k-nearest neighbors (kNN). Therefore, the proposed visual feature coding method well preserves similarity, locality and sparsity, and can be used to improve the dictionary itself in turn by proposed specially designed online learning scheme as well. Experiments on three well-recognized benchmark datasets have been conducted and the experimental results demonstrate that it achieves superior image classification performance to other state-of-the-art methods.
Image Classification Method by Combining Multi-features and Sparse Coding

LUO Hui-Lan,GUO Min-Jie,KONG Fan-Sheng

DOI: https://doi.org/10.3969/j.issn.1003-6059.2014.04.009

2014-01-01

Abstract:Using a single image feature to describe the image content is one-sided because of the insufficient information. Besides, the single coding method usually loses the spatial information. To solve these problems, an approach of integrating multi-features and sparse coding methods is proposed. Images are firstly divided into sub regions according to the spatial pyramid, and then the complementary advantages of scale invariant feature transform and the histogram of oriented gradients features are combined to produce various feature sets. Then, different clustering methods are used on different feature sets to acquire different codebooks. Next, two sparse coding methods, locality constrained linear coding and sparse coding based on each codebook are further employed respectively to get various image description sets. Finally, linear support vector machines are applied to image classification, and a voting method is used to determine the final classification. Experimental results show that the proposed method has good accuracy and robustness compared with some state-of-the-art methods.
A dictionary learning approach for classification: separating the particularity and the commonality

Shu Kong,Donghui Wang

DOI: https://doi.org/10.1007/978-3-642-33718-5_14

2012-01-01

Abstract:Empirically, we find that, despite the class-specific features owned by the objects appearing in the images, the objects from different categories usually share some common patterns, which do not contribute to the discrimination of them. Concentrating on this observation and under the general dictionary learning (DL) framework, we propose a novel method to explicitly learn a common pattern pool (the commonality) and class-specific dictionaries (the particularity) for classification. We call our method DL-COPAR, which can learn the most compact and most discriminative class-specific dictionaries used for classification. The proposed DL-COPAR is extensively evaluated both on synthetic data and on benchmark image databases in comparison with existing DL-based classification methods. The experimental results demonstrate that DL-COPAR achieves very promising performances in various applications, such as face recognition, handwritten digit recognition, scene classification and object recognition.
Online Discriminative Dictionary Learning for Image Classification Based on Block-Coordinate Descent Method

Shu Kong,Donghui Wang

DOI: https://doi.org/10.48550/arxiv.1203.0856

2012-01-01

Abstract:Previous researches have demonstrated that the framework of dictionary learning with sparse coding, in which signals are decomposed as linear combinations of a few atoms of a learned dictionary, is well adept to reconstruction issues. This framework has also been used for discrimination tasks such as image classification. To achieve better performances of classification, experts develop several methods to learn a discriminative dictionary in a supervised manner. However, another issue is that when the data become extremely large in scale, these methods will be no longer effective as they are all batch-oriented approaches. For this reason, we propose a novel online algorithm for discriminative dictionary learning, dubbed \textbf{ODDL} in this paper. First, we introduce a linear classifier into the conventional dictionary learning formulation and derive a discriminative dictionary learning problem. Then, we exploit an online algorithm to solve the derived problem. Unlike the most existing approaches which update dictionary and classifier alternately via iteratively solving sub-problems, our approach directly explores them jointly. Meanwhile, it can largely shorten the runtime for training and is also particularly suitable for large-scale classification issues. To evaluate the performance of the proposed ODDL approach in image recognition, we conduct some experiments on three well-known benchmarks, and the experimental results demonstrate ODDL is fairly promising for image classification tasks.
A Classification-Oriented Dictionary Learning Model: Explicitly Learning the Particularity and Commonality Across Categories.

Donghui Wang,Shu Kong

DOI: https://doi.org/10.1016/j.patcog.2013.08.004

IF: 8

2013-01-01

Pattern Recognition

Abstract:Empirically, we find that despite the most exclusively discriminative features owned by one specific object category, the various classes of objects usually share some common patterns, which do not contribute to the discrimination of them. Concentrating on this observation and motivated by the success of dictionary learning (DL) framework, in this paper, we propose to explicitly learn a class-specific dictionary (called particularity) for each category that captures the most discriminative features of this category, and simultaneously learn a common pattern pool (called commonality), whose atoms are shared by all the categories and only contribute to representation of the data rather than discrimination. In this way, the particularity differentiates the categories while the commonality provides the essential reconstruction for the objects. Thus, we can simply adopt a reconstruction-based scheme for classification. By reviewing the existing DL-based classification methods, we can see that our approach simultaneously learns a classification-oriented dictionary and drives the sparse coefficients as discriminative as possible. In this way, the proposed method will achieve better classification performance. To evaluate our method, we extensively conduct experiments both on synthetic data and real-world benchmarks in comparison with the existing DL-based classification algorithms, and the experimental results demonstrate the effectiveness of our method.
Multi-View Analysis Dictionary Learning for Image Classification.

Qianyu Wang,Yanqing Guo,Jiujun Wang,Xiangyang Luo,Xiangwei Kong

DOI: https://doi.org/10.1109/access.2018.2791578

IF: 3.9

2018-01-01

IEEE Access

Abstract:A great number of studies show that considering information from multiple views results in better performance than their single-view counterparts. However, many previous works aiming to improve classification performance fail to effectively tackle the inter-view correlation or the intra-class variability. Analysis dictionary learning (ADL) has theoretic significance and practical potential in classification tasks. Based on the ADL, this paper proposes a new method, namely, multi-view analysis dictionary learning (MvADL) for image classification. Specifically, multi-view analysis dictionaries are designed to reduce the intra-class variability in the transformed space in the multi-view scenario. Then, a marginalized classification term is incorporated to integrate the semantic information into the basic dictionary learning model. In the marginalized classification term, a marginalized target learning strategy is applied to improve the flexibility and discriminability of the whole model. Besides, an iteratively optimizing algorithm is designed to solve the proposed MvADL. Experiments on benchmark data sets demonstrate the superiority of our proposed method.
Learning attribute-aware dictionary for image classification and search

Junjie Cai,Zheng-Jun Zha,Huanbo Luan,Shiliang Zhang,Qi Tian

DOI: https://doi.org/10.1145/2461466.2461473

2013-01-01

Abstract:Bag-of-visual words (BoW) model has recently been well advocated for image classification and search. However, one critical limitation of existing BoW model is the lack of semantic information. To alleviate the impact of this issue, it is imperative to construct semantic-aware visual dictionary. In this paper, we propose a novel approach for learning visual word dictionary embedding intermediate-level semantics. Specifically, we first introduce an Attribute aware Dictionary Learning(AttrDL) scheme to learn multiple sub-dictionaries with specific semantic meanings. We divide training images into different sets and each represents a specific attribute. For each image set, an attribute-aware sub-vocabulary is learned. Hence, these resulting sub-vocabularies are more discriminative for semantics than the traditional vocabularies. Second, to get semantic-aware and discriminative BoW representation with the learned sub-vocabularies, we adopt the idea of L21-norm regularized sparse coding and recode the resulting sparse representation of each image. Experimental results show that the proposed scheme outperforms the state-of-the-art algorithms in both image classification and search tasks.
An Image Classification Method Based on Multiple Visual Dictionaries

Jian Zhang,Xiao-yin Zhang,Zhou Lei,Wen-feng Shen

DOI: https://doi.org/10.1109/csa.2015.18

2015-01-01

Abstract:In recent years, the bag of visual words (BoV) model is very popular in the field of computer vision. In particular, it has been widely used for image classification. The earliest method based on the BoV model, when constructing the visual dictionary, uses the clustering method to generate a single visual dictionary. This method does not consider the differences between categories of image dataset, leading to a poor performance of image classification. Later, an improved method is proposed that builds a visual dictionary for each category in the image dataset and then concatenates the visual dictionaries to produce the final visual dictionary. However, when there are many categories in the image dataset, using this method is time consuming. In addition, building a visual dictionary for each category leads to the visual words being too dense, losing the ability of distinguishing and expressing information and decreasing the performance of the image classification. In this paper, we proposed a new image classification method that is based on multiple visual dictionaries. This method addresses the above problems effectively and improves the performance of image classification significantly. To build a visual dictionary, the proposed image classification method first divides the image dataset containing many categories into N items randomly and uses the clustering method to cluster the local features of images of N items respectively. Next it generates a visual dictionary for each item in the image dataset. Finally it connects the visual dictionaries to produce the final visual dictionary of the entire image dataset. We conducted experiments on the caltech-101 and scene-67 image libraries and compared to the traditional methods. Our experimental results show that our proposed image classification method can match the best results published in the previous literatures in terms of classification accuracy rate.
Incorporating Visual Adjectives for Image Classification

Lingxi Xie,Jingdong Wang,Bo Zhang,Qi Tian

DOI: https://doi.org/10.1016/j.neucom.2015.12.008

IF: 6

2016-01-01

Neurocomputing

Abstract:Image classification is a fundamental problem in computer vision which implies a wide range of real-world applications. Conventional approaches for image classification often involve image description and training/testing phases. The Bag-of-Features (BoF) model is one of the most popular algorithms for image description, in which local descriptors are extracted, quantized, and summarized into global image representation.In the BoF model, all the visual descriptors are naturally treated as nouns, and plenty of useful contents are ignored. In this paper, we suggest to extract descriptive information, known as adjectives, to help visual recognition. We propose a simple framework to integrate various types of adjectives, i.e., color (or brightness), shape and location, for more powerful image representation. Experimental results on both scene recognition and fine-grained object recognition reveal that our approach achieves superior classification accuracy with reasonable computational overheads. It is also possible to generalize our model to many other multimedia applications such as large-scale image search.
Visual word coding based on difference maximization.

Yu-Bin Yang,Ling-Yan Pan,Yang Gao,Guang-Nan He,Yao Zhang

DOI: https://doi.org/10.1016/j.neucom.2012.08.050

IF: 6

2013-01-01

Neurocomputing

Abstract:Image classification is an important topic in computer vision, which becomes more and more challenging due to the rapid increase of the amounts and categories of images, as well as the different geometric deformations and illumination variations existing in image objects. “Bag-of-Features” model, also known as “Bag-of-Words” model, plays a fundamental and crucial role in generating efficient and discriminative image content representations by using local descriptors such as visual words, making it widely used in solving image classification problems. However, because of the weak discriminative power and strong ambiguity of the low-level visual features, the visual codebook, i.e., a set of visual words, generated in this model is usually over-completed and inconsistent for capturing image semantics. To address this issue, we propose a novel visual word coding algorithm in this paper based on difference maximization technique to improve the generated codebook model. Instead of mapping an image feature vector to one or multiple nearest visual words, the proposed approach utilizes a group of the nearest and the farthest visual words together in the coding process. Consequently, the representative variations of different image features are well kept and strengthened, which can then improve the discriminative power of the visual word descriptor significantly. We examine the performance of our visual word coding model extensively on four standard real-world image datasets, demonstrating that it captures image semantic content more accurately and achieves superior classification performance.
Bilevel Visual Words Coding for Image Classification

Jiemi Zhang,Chenxia Wu,Deng Cai,Jianke Zhu

2013-01-01

Abstract:Bag-of-Words approach has played an important role in recent works for image classification. In consideration of efficiency, most methods use k - means clustering to generate the codebook. The obtained codebooks often lose the cluster size and shape information with distortion errors and low discriminative power. Though some efforts have been made to optimize codebook in sparse coding, they usually incur higher computational cost. Moreover, they ignore the correlations between codes in the following coding stage, that leads to low discriminative power of the final representation. In this paper, we propose a bilevel visual words coding approach in consideration of representation ability, discriminative power and efficiency. In the bilevel codebook generation stage, k -means and an efficient spectral clustering are respectively run in each level by taking both class information and the shapes of each visual word cluster into account. To obtain discriminative representation in the coding stage, we design a certain localized coding rule with bilevel codebook to select local bases. To further achieve an efficient coding referring to this rule, an online method is proposed to efficiently learn a projection of local descriptor to the visual words in the codebook. After projection, coding can be efficiently completed by a low dimensional localized soft-assignment. Experimental results show that our proposed bilevel visual words coding approach outperforms the state-of-the-art approaches for image classification.
Learning dictionary on manifolds for image classification

LiuBao-Di,WangYu-Xiong,ZhangYu-Jin,ShenBin

IF: 8

2013-01-01

Pattern Recognition

Abstract:At present, dictionary based models have been widely used in image classification. The image features are approximated as a linear combination of bases selected from the dictionary in a sparse spac...
Feature discovering for image classification via wavelet-like pattern decomposition.

Yurui Xie,Hongliang Li,Chao Huang,Bo Wu,Linfeng Xu

DOI: https://doi.org/10.1016/j.jvcir.2016.08.002

2016-01-01

Abstract:A feature discovering method based on dictionary learning is proposed.Representative visual samples can be exploited by the proposed objective.The commonality and individuality visual patterns are decomposed jointly.The discrimination of feature is improved using a wavelet-like architecture. In this paper, we propose a feature discovering method incorporated with a wavelet-like pattern decomposition strategy to address the image classification problem. In each level, we design a discriminative feature discovering dictionary learning (DFDDL) model to exploit the representative visual samples from each class and further decompose the commonality and individuality visual patterns simultaneously. The representative samples reflect the discriminative visual cues per class, which are beneficial for the classification task. Furthermore, the commonality visual elements capture the communal visual patterns across all classes. Meanwhile, the class-specific discriminative information can be collected by the learned individuality visual elements. To further discover the more discriminative feature information from each class, we then integrate the DFDDL into a wavelet-like hierarchical architecture. Due to the designed hierarchical strategy, the discriminative power of feature representation can be promoted. In the experiment, the effectiveness of proposed method is verified on the challenging public datasets.
Visual language modeling for image classification.

Lei Wu,Mingjing Li,Zhiwei Li,Wei-Ying Ma,Nenghai Yu

DOI: https://doi.org/10.1145/1290082.1290101

2007-01-01

Abstract:ABSTRACTAlthough it has been studied for many years, image classification is still a challenging problem. In this paper, we propose a visual language modeling method for content-based image classification. It transforms each image into a matrix of visual words, and assumes that each visual word is conditionally dependent on its neighbors. For each image category, a visual language model is constructed using a set of training images, which captures both the co-occurrence and proximity information of visual words. According to how many neighbors are taken in consideration, three kinds of language models can be trained, including unigram, bigram and trigram, each of which corresponds to a different level of model complexity. Given a test image, its category is determined by estimating how likely it is generated under a specific category. Compared with traditional methods that are based on bag-of-words models, the proposed method can utilize the spatial correlation of visual words effectively in image classification. In addition, we propose to use the absent words, which refer to those appearing frequently in a category but not in the target image, to help image classification. Experimental results show that our method can achieve comparable accuracy while performing classification much more quickly.
Unifying Discriminative Visual Codebook Generation with Classifier Training for Object Category Recognition

Liu Yang,Rong Jin,Rahul Sukthankar,Frederic Jurie

DOI: https://doi.org/10.1109/cvpr.2008.4587504

2008-01-01

Abstract:The idea of representing images using a bag of visual words is currently popular in object category recognition. Since this representation is typically constructed using unsupervised clustering, the resulting visual words may not capture the desired information. Recent work has explored the construction of discriminative visual codebooks that explicitly consider object category information. However, since the codebook generation process is still disconnected from that of classifier training, the set of resulting visual words, while individually discriminative, may not be those best suited for the classifier This paper proposes a novel optimization framework that unifies codebook generation with classifier training. In our approach, each image feature is encoded by a sequence of "visual bits" optimized for each category. An image, which can contain objects from multiple categories, is represented using aggregates of visual bits for each category. Classifiers associated with different categories determine how well a given image corresponds to each category. Based on the performance of these classifiers on the training data, we augment the visual words by generating additional bits. The classifiers are then updated to incorporate the new representation. These two phases are repeated until the desired performance is achieved. Experiments compare our approach to standard clustering-based methods and with state-of-the-art discriminative visual codebook generation. The significant improvements over previous techniques clearly demonstrate the value of unifying representation and classification into a single optimization framework.
Learning Discriminative Visual Dictionary for Natural Scene Categorization

Ying Huang,Wenmin Wang,Ronggang Wang

DOI: https://doi.org/10.1109/icassp.2015.7178186

2015-01-01

ICASSP

Abstract:Many successful systems for scene recognition transform low-level descriptors into complex representations. This process consists of the two steps: 1) feature coding, which performs a pointwise transformation of the descriptors into a representation adapted to the task, and 2) image pooling, which summarizes the coded features. Even though these two steps have been paid so much attention, but there are still some problems in combining scene semantic with local features. The goal of this paper is threefold: to address the problem by modifying the traditional bag-of-features (BoF) framework; to show how to achieve the best performance by learning a semi-supervised discriminative dictionary; and to provide theoretical and empirical insight into the remarkable performance. By teasing apart components shared by modern scene categorization pipeline, our approach aims to facilitate the design of better scene recognition architectures.
From visual words to a visual grammar: using language modelling for image classification

Antonio Foncubierta-Rodríguez,Henning Müller,Adrien Depeursinge

DOI: https://doi.org/10.48550/arXiv.1703.05571

2017-03-16

Computer Vision and Pattern Recognition

Abstract:The Bag--of--Visual--Words (BoVW) is a visual description technique that aims at shortening the semantic gap by partitioning a low--level feature space into regions of the feature space that potentially correspond to visual concepts and by giving more value to this space. In this paper we present a conceptual analysis of three major properties of language grammar and how they can be adapted to the computer vision and image understanding domain based on the bag of visual words paradigm. Evaluation of the visual grammar shows that a positive impact on classification accuracy and/or descriptor size is obtained when the technique are applied when the proposed techniques are applied.
Large Visual Words For Large Scale Image Classification

Sheng Tang,Hui Chen,Ke Lv,Yong-Dong Zhang

DOI: https://doi.org/10.1109/ICIP.2015.7350984

2015-01-01

Abstract:Recently, using large visual vocabulary or codebooks to quantize and partition the set of local feature descriptors into large set of disjoint subsets termed visual words (or large visual words) has become an important research topic in solving many computer vision problems including near duplicate image retrieval, object retrieval, etc. Generally, large visual words means a heavy burden on the cost of time and memory space for both the construction of large vocabulary and the searching process, especially for large scale applications. In this paper, we present an efficient generation approach of large visual words with a very compact vocabulary, namely two dictionaries learned with sparse non-negative matrix factorization (NMF). After piecewise sparse decomposition of features with two learned dictionaries, we map a pair of indices of the dictionary's bases corresponding to the maximum elements of the two sparse codes to a large set of visual words upon the assumption that data with similar properties will share the same base with the largest sparse coefficient. With the help of an inverted file structure built through the large visual words, K-nearest neighbors (KNN) can be efficiently retrieved. Therefore, we can classify images very efficiently with the incorporation of our fast KNN search based on large visual words into SVM-KNN method. Experiments on the public Oxford dataset, and ACM Multimedia 2013 Yahoo! image classification challenge dataset show that our approach is both effective and efficient.
Object Recognition via Adaptive Multi-level Feature Integration

Mei Wang,Yanling Wu,Guangda Li,Xiangdong Zhou

DOI: https://doi.org/10.1109/APWeb.2010.24

2010-01-01

Abstract:Object category recognition is a challenging task due to the low level and non-discrimination in visual representation. Most previous methods concentrate to find better high level visual features. Recently, optimally integrating various features to solve the problem attracted more interests. In this paper, we provide a novel method for object category recognition by improving the popular bag-of-words (BoW) methods from the following two aspects. First, we propose to extract a series of high level visual features which exploit both the local spatial co occurrence between low level visual words and the global spatial layout of the object parts. To obtain the global spatial features, a fast method is proposed to generate the semantic meaningful object parts by exploiting the geometric position distribution of the local salient regions. The image part patches are further quantized as semantic coherent high level visual words by using correlational spectral clustering. Based on it, simplified 2D string representation is introduced to model the global spatial patterns of the objects. Second, a multi-kernel learning framework is proposed to adaptively integrate extracted features in an optimal way. For each object class, an optimal feature weight coefficient is learned automatically and separately to combine both the low level and high level visual features by considering their contribution for the different object class. The tests on Caltech-101 and Pascal- VOC 06 dataset demonstrated that our method outperforms the baseline method BoW and state-of-the-art Multi-CM model .
Modified Bag of Visual Words Model for Image Classification

Zainab N. Sultani,Ban N. Dhannoon,,

DOI: https://doi.org/10.22401/anjs.24.2.11

2021-06-01

Abstract:Image classification is acknowledged as one of the most critical and challenging tasks in computer vision. The bag of visual words (BoVW) model has proven to be very efficient for image classification tasks since it can effectively represent distinctive image features in vector space. In this paper, BoVW using Scale-Invariant Feature Transform (SIFT) and Oriented Fast and Rotated BRIEF(ORB) descriptors are adapted for image classification. We propose a novel image classification system using image local feature information obtained from both SIFT and ORB local feature descriptors. As a result, the constructed SO-BoVW model presents highly discriminative features, enhancing the classification performance. Experiments on Caltech-101 and flowers dataset prove the effectiveness of the proposed method.

Visual feature coding for image classification integrating dictionary structure

Visual Feature Coding for Image Classification Integrating Dictionary Structure

Image Classification Method by Combining Multi-features and Sparse Coding

A dictionary learning approach for classification: separating the particularity and the commonality

Online Discriminative Dictionary Learning for Image Classification Based on Block-Coordinate Descent Method

A Classification-Oriented Dictionary Learning Model: Explicitly Learning the Particularity and Commonality Across Categories.

Multi-View Analysis Dictionary Learning for Image Classification.

Learning attribute-aware dictionary for image classification and search

An Image Classification Method Based on Multiple Visual Dictionaries

Incorporating Visual Adjectives for Image Classification

Visual word coding based on difference maximization.

Bilevel Visual Words Coding for Image Classification

Learning dictionary on manifolds for image classification

Feature discovering for image classification via wavelet-like pattern decomposition.

Visual language modeling for image classification.

Unifying Discriminative Visual Codebook Generation with Classifier Training for Object Category Recognition

Learning Discriminative Visual Dictionary for Natural Scene Categorization

From visual words to a visual grammar: using language modelling for image classification

Large Visual Words For Large Scale Image Classification

Object Recognition via Adaptive Multi-level Feature Integration

Modified Bag of Visual Words Model for Image Classification