Abstract:Deriving from the gradient vector of a generative model of local features, Fisher vector coding (FVC) has been identified as an effective coding method for image classification. Most, if not all, FVC implementations employ the Gaussian mixture model (GMM) as the generative model for local features. However, the representative power of a GMM can be limited because it essentially assumes that local features can be characterized by a fixed number of feature prototypes, and the number of prototypes is usually small in FVC. To alleviate this limitation, in this work, we break the convention which assumes that a local feature is drawn from one of a few Gaussian distributions. Instead, we adopt a compositional mechanism which assumes that a local feature is drawn from a Gaussian distribution whose mean vector is composed as a linear combination of multiple key components, and the combination weight is a latent random variable. In doing so we greatly enhance the representative power of the generative model underlying FVC. To implement our idea, we design two particular generative models following this compositional approach. In our first model, the mean vector is sampled from the subspace spanned by a set of bases and the combination weight is drawn from a Laplace distribution. In our second model, we further assume that a local feature is composed of a discriminative part and a residual part. As a result, a local feature is generated by the linear combination of discriminative part bases and residual part bases. The decomposition of the discriminative and residual parts is achieved via the guidance of a pre-trained supervised coding method. By calculating the gradient vector of the proposed models, we derive two new Fisher vector coding strategies. The first is termed Sparse Coding-based Fisher Vector Coding (SCFVC) and can be used as the substitute of traditional GMM based FVC. The second is termed Hybrid Sparse Coding-based Fisher vector coding (HSCFVC) since it combines the merits of both pre-trained supervised coding methods and FVC. Using pre-trained Convolutional Neural Network (CNN) activations as local features, we experimentally demonstrate that the proposed methods are superior to traditional GMM based FVC and achieve state-of-the-art performance in various image classification tasks.

Color Multi-Fusion Fisher Vector Feature for Fine Art Painting Categorization and Influence Analysis.

Sparse Representation Based Complete Kernel Marginal Fisher Analysis Framework for Computational Art Painting Categorization

Sentiment Analysis Using Deep Robust Complementary Fusion of Multi-Features and Multi-Modalities.

MTFFNet: a Multi-task Feature Fusion Framework for Chinese Painting Classification

A Novel Multi-Feature Fusion Method in Merging Information of Heterogenous-View Data for Oil Painting Image Feature Extraction and Recognition

Author Recognition of Fine-Art Paintings

Deep Feature Fusion for High-Resolution Aerial Scene Classification

MCCFNet: Multi-channel Color Fusion Network For Cognitive Classification of Traditional Chinese Paintings

Multi-Feature Fusion Image Retrieval Algorithm Based on Fuzzy Color

Spatial Weighted Fisher Vector for Image Retrieval

Encoding High Dimensional Local Features By Sparse Coding Based Fisher Vectors

A Sparse Representation Model Using the Complete Marginal Fisher Analysis Framework and Its Applications to Visual Recognition

Compositional Model Based Fisher Vector Coding for Image Classification

A fuzzy control algorithm based on artificial intelligence for the fusion of traditional Chinese painting and AI painting

A data-driven study of image feature extraction and fusion

UMAIR-FPS: User-aware Multi-modal Animation Illustration Recommendation Fusion with Painting Style

Image Retrieval Based on Multi-feature Fusion of Color Connected Region

Structure-Aware Multimodal Feature Fusion for RGB-D Scene Classification and Beyond

Inheritable Fisher Vector Feature for Kinship Verification.

Modality And Component Aware Feature Fusion For Rgb-D Scene Classification

Dif-Fusion: Toward High Color Fidelity in Infrared and Visible Image Fusion With Diffusion Models