Abstract:This paper addresses the problem of very large-scale image retrieval, focusing on improving its accuracy and robustness. We target enhanced robustness of search to factors, such as variations in illumination, object appearance and scale, partial occlusions, and cluttered backgrounds—particularly important when a search is performed across very large datasets with significant variability. We propose a novel CNN-based global descriptor, called REMAP, which learns and aggregates a hierarchy of deep features from multiple CNN layers, and is trained end-to-end with a triplet loss. REMAP explicitly learns discriminative features which are mutually supportive and complementary at various semantic levels of visual abstraction. These dense local features are max-pooled spatially at each layer, within multi-scale overlapping regions, before aggregation into a single image-level descriptor. To identify the semantically useful regions and layers for retrieval, we propose to measure the information gain of each region and layer using KL-divergence. Our system effectively learns during training how useful various regions and layers are and weights them accordingly. We show that such relative entropy-guided aggregation outperforms classical CNN-based aggregation controlled by SGD. The entire framework is trained in an end-to-end fashion, outperforming the latest state-of-the-art results. On image retrieval datasets Holidays, Oxford, and MPEG, the REMAP descriptor achieves mAP of 95.5%, 91.5%, and 80.1%, respectively, outperforming any results published to date. REMAP also formed the core of the winning submission to the Google Landmark Retrieval Challenge on Kaggle.

Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval

Searching Parameterized AP Loss for Object Detection

Robust and Decomposable Average Precision for Image Retrieval

Score-ranking smooth average precision loss for remote sensing image retrieval

Hierarchical Average Precision Training for Pertinent Image Retrieval

Deep Scale-Aware Image Smoothing

Image Smoothing via Multiscale Global Perception

REMAP: Multi-Layer Entropy-Guided Pooling of Dense CNN Features for Image Retrieval

Not All Pairs are Equal: Hierarchical Learning for Average-Precision-Oriented Video Retrieval

AP-Loss for Accurate One-Stage Object Detection

A Deep Evaluator for Image Retargeting Quality by Geometrical and Contextual Interaction

From Document to Image: Learning a Scalable Ranking Model for Content Based Image Retrieval

Image smoothing via unsupervised learning

Pixel-Level Non-local Image Smoothing With Objective Evaluation

Real-time Image Smoothing via Iterative Least Squares

Scalable Person Re-identification on Supervised Smoothed Manifold

FAPP: Extremely Fast Approach to Boosting Image Matching Precision

Scalable Learning of Non-Decomposable Objectives

GSmooth: Certified Robustness against Semantic Transformations via Generalized Randomized Smoothing

Revisiting Deep Image Smoothing and Intrinsic Image Decomposition.