Abstract:This paper provides a unified view of loss functions used in learning to rank. Loss function is a key component in learning to rank, because it encodes human knowledge on evaluation of ranking and guides the process of learning. Many loss functions have been proposed in the literature of learning to rank, with different forms and different motivations, and have been exploited in the development of various algorithms. However, there are several questions not well answered yet regarding the loss functions: (i) what is the relationship among them? (ii) can the optimization of them lead to the maximization of IR measures such as NDCG? (iii) how can we modify them to improve the performance of learning algorithms? In this paper, we try to answer these questions by proposing a quantity named ‘the unified loss for ranking’. The unified loss is defined by modeling ranking as a sequence of classification tasks, and assigning different weights to the tasks to reflect their importance in the overall ranking problem. With regards to the unified loss, we obtain the following three theoretical results. First, the loss functions in most existing algorithms, such as Ranking SVM, RankNet, RankBoost, and ListMLE, are margin-based surrogate losses (and also upper bounds) of the unified loss with uniform weight. Second, the unified loss is an upper bound of one minus NDCG, when the weights are set according to the position discount function and the gain function in NDCG. Third, if we can modify the loss functions of existing algorithms by introducing the weights in the unified loss, they are still upper bounds of the corresponding unified loss, and thus upper bounds of one minus NDCG. According to the results, we can expect that the minimization of the modified loss functions will lead to the effective maximization of NDCG. We have performed experiments on benchmark datasets to validate our theoretical findings. The results show that the ranking performances of the models trained with the modified loss functions can be much better than those of the models trained with the original loss functions.

Listwise learning to rank method combining approximate NDCG ranking indicator with Conditional Generative Adversarial Networks

Online Learning to Rank in a Listwise Approach for Information Retrieval

Extended Missing Data Imputation via GANs for Ranking Applications

Learning to Rank in Generative Retrieval

List-aware Reranking-Truncation Joint Model for Search and Retrieval-augmented Generation

A Unified View of Loss Functions in Learning to Rank

Learning to Rank Ace Neural Architectures via Normalized Discounted Cumulative Gain

UGC: Unified GAN Compression for Efficient Image-to-Image Translation

Ranked List Loss for Deep Metric Learning

Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Search

Large-scale Stochastic Optimization of NDCG Surrogates for Deep Learning with Provable Convergence

Learning Multimodal Neural Network with Ranking Examples

GS2P: a generative pre-trained learning to rank model with over-parameterization for web-scale search

Learning To Rank Resources with GNN

A Theoretical Analysis of NDCG Type Ranking Measures.

Optimizing Preference Alignment with Differentiable NDCG Ranking

GNNRank: Learning Global Rankings from Pairwise Comparisons via Directed Graph Neural Networks

RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolution

An Alternative Cross Entropy Loss for Learning-to-Rank

Listwise Generative Retrieval Models via a Sequential Learning Process

Enhancing sequential recommendation with contrastive Generative Adversarial Network