Abstract:The MPEG compact descriptors for visual search (CDVS) is a standard toward image matching and retrieval. To achieve high retrieval accuracy over a large scale image/video dataset, recent research efforts have demonstrated that employing extremely high-dimensional descriptors such as the Fisher vector (FV) and the vector of locally aggregated descriptors (VLAD) can yield good performance. Since the FV (or VLAD) possesses high discriminability but small visual vocabulary, it has been adopted by CDVS to construct a global compact descriptor. In this paper, we study the development of global compact descriptors in the completed CDVS standard and the emerging compact descriptors for video analysis (CDVA) standard, in which we formulate the FV (or VLAD) compression as a resource-constrained optimization problem. Accordingly, we propose a codebook-free aggregation method via dual selection to generate a global compact visual descriptor, which supports fast and accurate feature matching free of large visual codebooks, fulfilling the low memory requirement of mobile visual search at significantly reduced latency. Specifically, we investigate both sample-specific Gaussian component redundancy and bit dependency within a binary aggregated descriptor to produce compact binary codes. Our technique contributes to the scalable compressed Fisher vector (SCFV) adopted by the CDVS standard. Moreover, the SCFV descriptor is currently serving as the frame-level hand-crafted video feature, which inspires the inheritance of CDVS descriptors for the emerging CDVA standard. Furthermore, we investigate the positive complementary effect of our standard compliant compact descriptor and deep learning based features extracted from convolutional neural networks with significant mean average precision gains. Extensive evaluation over benchmark databases shows the significant merits of the codebook-free binary codes for scalable visual search.

Codebook Enhancement of Vlad Representation for Visual Recognition.

Shrinking Encoding with Two-Level Codebook Learning for Fine-Grained Fish Recognition

Boosting VLAD with Supervised Dictionary Learning and High-Order Statistics.

Visual word coding based on difference maximization.

Codebook-Free Compact Descriptor for Scalable Visual Search.

A Simple but Efficient Way to Combine Vlad with Locality-Constrained Linear Coding

Unifying Discriminative Visual Codebook Generation with Classifier Training for Object Category Recognition

Bilevel Visual Words Coding for Image Classification

A Fast Algorithm For Creating A Compact And Discriminative Visual Codebook

Image Representation Optimization Based on Locally Aggregated Descriptors.

Improvements in image categorization using codebook ensembles

Improved Cluster Center Adaption for Image Classification

Visual Recognition Using Density Adaptive Clustering

VQ Codebook Design Using Modified K-means Algorithm with Feature Classification and Grouping Based Initialization

Building Descriptive and Discriminative Visual Codebook for Large-Scale Image Applications.

Efficient and Effective Visual Codebook Generation Using Additive Kernels

Improving VLAD with Regional PCA Whitening.

Learning visual codebooks for image classification using spectral clustering

Online Clustered Codebook

Fine-residual VLAD for Image Retrieval.

VLAD-VSA: Cross-Domain Face Presentation Attack Detection with Vocabulary Separation and Adaptation.