Abstract:This paper provides a unified view of loss functions used in learning to rank. Loss function is a key component in learning to rank, because it encodes human knowledge on evaluation of ranking and guides the process of learning. Many loss functions have been proposed in the literature of learning to rank, with different forms and different motivations, and have been exploited in the development of various algorithms. However, there are several questions not well answered yet regarding the loss functions: (i) what is the relationship among them? (ii) can the optimization of them lead to the maximization of IR measures such as NDCG? (iii) how can we modify them to improve the performance of learning algorithms? In this paper, we try to answer these questions by proposing a quantity named ‘the unified loss for ranking’. The unified loss is defined by modeling ranking as a sequence of classification tasks, and assigning different weights to the tasks to reflect their importance in the overall ranking problem. With regards to the unified loss, we obtain the following three theoretical results. First, the loss functions in most existing algorithms, such as Ranking SVM, RankNet, RankBoost, and ListMLE, are margin-based surrogate losses (and also upper bounds) of the unified loss with uniform weight. Second, the unified loss is an upper bound of one minus NDCG, when the weights are set according to the position discount function and the gain function in NDCG. Third, if we can modify the loss functions of existing algorithms by introducing the weights in the unified loss, they are still upper bounds of the corresponding unified loss, and thus upper bounds of one minus NDCG. According to the results, we can expect that the minimization of the modified loss functions will lead to the effective maximization of NDCG. We have performed experiments on benchmark datasets to validate our theoretical findings. The results show that the ranking performances of the models trained with the modified loss functions can be much better than those of the models trained with the original loss functions.

Construction Algorithm of ranking Model Based on Non-Convex Upper Bound

A new ranking model constructing algorithm based on direct optimization of NDCG

RANK-META: A two-stage ranking algorithm based on meta-learning

Directly optimizing evaluation measures in learning to rank.

An Efficient Method for Convex Constrained Rank Minimization Problems Based on Dc Programming

Ranking Measures and Loss Functions in Learning to Rank.

Efficient Recovery of Low-Rank Matrix Via Double Nonconvex Nonsmooth Rank Minimization.

Large-scale Stochastic Optimization of NDCG Surrogates for Deep Learning with Provable Convergence

Large-Scale Affine Matrix Rank Minimization with a Novel Nonconvex Regularizer.

A Theoretical Analysis of NDCG Type Ranking Measures.

Creating an Acceptable Consensus Ranking for Group Decision Making

New ranking method for many-obj ective problems

Improved Nonconvex Optimization Model for Low-Rank Matrix Recovery

Offline Model-Based Optimization by Learning to Rank

RESEARCH ON ALGORITHM OF INEQUALITY CONSTRAINED RANK DEFICIENT ADJUSTMENT

Adaptive and efficient high-order rating distance optimization model with slack variable

A Unified View of Loss Functions in Learning to Rank

Robust Optimization with Convex Model Considering Bounded Constraints on Performance Variation

A New Nonconvex Rank Approximation of RPCA

Robust Recovery of Low-Rank Matrices Via Non-Convex Optimization

Xorder: A Model Agnostic Post-Processing Framework for Achieving Ranking Fairness While Maintaining Algorithm Utility.