Abstract:The objective of deep metric learning (DML) is to learn embeddings that can capture semantic similarity and dissimilarity information among data points. Existing pairwise or tripletwise loss functions used in DML are known to suffer from slow convergence due to a large proportion of trivial pairs or triplets as the model improves. To improve this, ranking-motivated structured losses are proposed recently to incorporate multiple examples and exploit the structured information among them. They converge faster and achieve state-of-the-art performance. In this work, we unveil two limitations of existing ranking-motivated structured losses and propose a novel ranked list loss to solve both of them. First, given a query, only a fraction of data points is incorporated to build the similarity structure. Consequently, some useful examples are ignored and the structure is less informative. To address this, we propose to build a set-based similarity structure by exploiting all instances in the gallery. The learning setting can be interpreted as few-shot retrieval: given a mini-batch, every example is iteratively used as a query, and the rest ones compose the gallery to search, i.e., the support set in few-shot setting. The rest examples are split into a positive set and a negative set. For every mini-batch, the learning objective of ranked list loss is to make the query closer to the positive set than to the negative set by a margin. Second, previous methods aim to pull positive pairs as close as possible in the embedding space. As a result, the intraclass data distribution tends to be extremely compressed. In contrast, we propose to learn a hypersphere for each class in order to preserve useful similarity structure inside it, which functions as regularisation. Extensive experiments demonstrate the superiority of our proposal by comparing with the state-of-the-art methods on the fine-grained image retrieval task. Our source code is available online: https://github.com/XinshaoAmosWang/Ranked-List-Loss-for-DML.

Joint Structural Learning to Rank with Deep Linear Feature Learning

Multimedia Retrieval Via Deep Learning to Rank

Cross-Modal Learning to Rank Via Latent Joint Representation

Joint Learning of Latent Similarity and Local Embedding for Multi-View Clustering

Learning Bregman Distance Functions for Structural Learning to Rank.

Simple to Complex Cross-modal Learning to Rank

Learning to rank relational objects and its application to web search.

Ranked List Loss for Deep Metric Learning

Scalable Multimedia Retrieval By Deep Learning Hashing With Relative Similarity Learning

Joint Embedding Learning and Low-Rank Approximation: A Framework for Incomplete Multi-view Learning

From Document to Image: Learning a Scalable Ranking Model for Content Based Image Retrieval

Learning to Combine Ad-hoc Ranking Functions for Image Retrieval

Structural Bregman Distance Functions Learning to Rank with Self-Reinforcement

Deep Compositional Cross-modal Learning to Rank via Local-Global Alignment

Dual low-rank structure embedding for robust visual information processing

Learning Cross-Media Joint Representation with Sparse and Semisupervised Regularization

Structural Learning of Diverse Ranking.

Joint Representation Learning for Top-N Recommendation with Heterogeneous Information Sources

A Low Rank Structural Large Margin Method for Cross-Modal Ranking

Learning Spatially Variant Linear Representation Models for Joint Filtering

Deep Metric Learning with Self-Supervised Ranking.