Abstract:The objective of deep metric learning (DML) is to learn embeddings that can capture semantic similarity and dissimilarity information among data points. Existing pairwise or tripletwise loss functions used in DML are known to suffer from slow convergence due to a large proportion of trivial pairs or triplets as the model improves. To improve this, ranking-motivated structured losses are proposed recently to incorporate multiple examples and exploit the structured information among them. They converge faster and achieve state-of-the-art performance. In this work, we unveil two limitations of existing ranking-motivated structured losses and propose a novel ranked list loss to solve both of them. First, given a query, only a fraction of data points is incorporated to build the similarity structure. Consequently, some useful examples are ignored and the structure is less informative. To address this, we propose to build a set-based similarity structure by exploiting all instances in the gallery. The learning setting can be interpreted as few-shot retrieval: given a mini-batch, every example is iteratively used as a query, and the rest ones compose the gallery to search, i.e., the support set in few-shot setting. The rest examples are split into a positive set and a negative set. For every mini-batch, the learning objective of ranked list loss is to make the query closer to the positive set than to the negative set by a margin. Second, previous methods aim to pull positive pairs as close as possible in the embedding space. As a result, the intraclass data distribution tends to be extremely compressed. In contrast, we propose to learn a hypersphere for each class in order to preserve useful similarity structure inside it, which functions as regularisation. Extensive experiments demonstrate the superiority of our proposal by comparing with the state-of-the-art methods on the fine-grained image retrieval task. Our source code is available online: https://github.com/XinshaoAmosWang/Ranked-List-Loss-for-DML.

Score-ranking smooth average precision loss for remote sensing image retrieval

Learning Local Feature Descriptors Through Ranking Losses Improved by Variance Shrinkage

Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval

Fast Low Rank Representation Based Spatial Pyramid Matching for Image Classification.

Ranked List Loss for Deep Metric Learning

Deep Multi-Similarity Hashing with Spatial-Enhanced Learning for Remote Sensing Image Retrieval

Remote Sensing Single-Image Resolution Improvement Using A Deep Gradient-Aware Network with Image-Specific Enhancement

CS-Net: Deep Multibranch Network Considering Scene Features for Sharpness Assessment of Remote Sensing Images

Sphere Loss: Learning Discriminative Features for Scene Classification in a Hyperspherical Feature Space

Not All Pairs are Equal: Hierarchical Learning for Average-Precision-Oriented Video Retrieval

Learning to rerank images with enhanced spatial verification

A Comparative Study of Deep Learning Loss Functions for Multi-Label Remote Sensing Image Classification

Directly optimizing evaluation measures in learning to rank.

Similarity and Ranking Preserving Deep Hashing for Image Retrieval.

Robust and Decomposable Average Precision for Image Retrieval

Deep Metric Learning with Self-Supervised Ranking.

Proxy-Based Rotation Invariant Deep Metric Learning for Remote Sensing Image Retrieval

Enhancing Remote Sensing Image Retrieval with Triplet Deep Metric Learning Network

RSDS: A Specialized Loss Calculation Method for Dense Small Object Detection in Remote Sensing Images

A Learnable Joint Spatial and Spectral Transformation for High Resolution Remote Sensing Image Retrieval

Soml: Sparse Online Metric Learning with Application to Image Retrieval