Abstract:Deriving from the gradient vector of a generative model of local features, Fisher vector coding (FVC) has been identified as an effective coding method for image classification. Most, if not all, FVC implementations employ the Gaussian mixture model (GMM) as the generative model for local features. However, the representative power of a GMM can be limited because it essentially assumes that local features can be characterized by a fixed number of feature prototypes, and the number of prototypes is usually small in FVC. To alleviate this limitation, in this work, we break the convention which assumes that a local feature is drawn from one of a few Gaussian distributions. Instead, we adopt a compositional mechanism which assumes that a local feature is drawn from a Gaussian distribution whose mean vector is composed as a linear combination of multiple key components, and the combination weight is a latent random variable. In doing so we greatly enhance the representative power of the generative model underlying FVC. To implement our idea, we design two particular generative models following this compositional approach. In our first model, the mean vector is sampled from the subspace spanned by a set of bases and the combination weight is drawn from a Laplace distribution. In our second model, we further assume that a local feature is composed of a discriminative part and a residual part. As a result, a local feature is generated by the linear combination of discriminative part bases and residual part bases. The decomposition of the discriminative and residual parts is achieved via the guidance of a pre-trained supervised coding method. By calculating the gradient vector of the proposed models, we derive two new Fisher vector coding strategies. The first is termed Sparse Coding-based Fisher Vector Coding (SCFVC) and can be used as the substitute of traditional GMM based FVC. The second is termed Hybrid Sparse Coding-based Fisher vector coding (HSCFVC) since it combines the merits of both pre-trained supervised coding methods and FVC. Using pre-trained Convolutional Neural Network (CNN) activations as local features, we experimentally demonstrate that the proposed methods are superior to traditional GMM based FVC and achieve state-of-the-art performance in various image classification tasks.

Sparse coding based Fisher vector using a Bayesian approach

Sparse Bayesian Learning Approach for Discrete Signal Reconstruction

Encoding High Dimensional Local Features By Sparse Coding Based Fisher Vectors

Compositional Model Based Fisher Vector Coding for Image Classification

Model-based sparse coding beyond Gaussian independent model

Modelling local and global phenomena with sparse Gaussian processes

Covariance-Free Sparse Bayesian Learning

Probabilistic Feature Selection and Classification Vector Machine

Sparse Bayesian Approach for Feature Selection

Bayesian High-dimensional Linear Regression with Sparse Projection-posterior

Efficient Image Categorization with Sparse Fisher Vector.

Efficient Sparsity Estimation Via Marginal-Lasso Coding.

Model-based Clustering with Sparse Covariance Matrices

Efficient, multimodal, and derivative-free Bayesian inference with Fisher-Rao gradient flows

Computationally Efficient Sparse Bayesian Learning Via Generalized Approximate Message Passing

Sparse Modified Marginal Fisher Analysis for Facial Expression Recognition

Scalable GAM using sparse variational Gaussian processes

Compressed Sensing Using Generative Models Based on Fisher Information

Bayesian Non-linear Latent Variable Modeling via Random Fourier Features

Vector Approximate Message Passing with Sparse Bayesian Learning for Gaussian Mixture Prior

Image Classification with CNN-based Fisher Vector Coding