Abstract:Unbiased learning to rank has been proposed to alleviate the biases in the search ranking, making it possible to train ranking models with user interaction data. In real applications, search engines are designed to display only the most relevant k documents from the retrieved candidate set. The rest candidates are discarded. As a consequence, position bias and sample selection bias usually occur simultaneously. Existing unbiased learning to rank approaches either focus on one type of bias (e.g., position bias) or mitigate the position bias and sample selection bias with separate components, overlooking their associations. In this study, we first analyze the mechanisms and associations of position bias and sample selection bias from the viewpoint of a causal graph. Based on the analysis, we propose Causal Likelihood Decomposition (CLD), a unified approach to simultaneously mitigating these two biases in top-k learning to rank. By decomposing the log-likelihood of the biased data as an unbiased term that only related to relevance, plus other terms related to biases, CLD successfully detaches the relevance from position bias and sample selection bias. An unbiased ranking model can be obtained from the unbiased term, via maximizing the whole likelihood. An extension to the pairwise neural ranking is also developed. Advantages of CLD include theoretical soundness and a unified framework for pointwise and pairwise unbiased top-k learning to rank. Extensive experimental results verified that CLD, including its pairwise neural extension, outperformed the baselines by mitigating both the position bias and the sample selection bias. Empirical studies also showed that CLD is robust to the variation of bias severity and the click noise.

Debiasing Learning to Rank Models with Generative Adversarial Networks

Model-based Unbiased Learning to Rank

Unbiased Learning to Rank with Unbiased Propensity Estimation

DebiasGAN: Eliminating Position Bias in News Recommendation with Adversarial Learning

LBD: Decouple Relevance and Observation for Individual-Level Unbiased Learning to Rank

Unbiased Learning-to-Rank with Biased Feedback

Towards Disentangling Relevance and Bias in Unbiased Learning to Rank

Scalar is Not Enough: Vectorization-based Unbiased Learning to Rank

Analysis of Multivariate Scoring Functions for Automatic Unbiased Learning to Rank

Scalar is Not Enough

Whole Page Unbiased Learning to Rank

Unbiased Learning to Rank

Unbiased Learning-to-Rank Needs Unconfounded Propensity Estimation

Unbiased Top-k Learning to Rank with Causal Likelihood Decomposition

Unconfounded Propensity Estimation for Unbiased Ranking

Contextual Dual Learning Algorithm with Listwise Distillation for Unbiased Learning to Rank

A Deep Recurrent Survival Model for Unbiased Ranking

Unbiased Learning to Rank with Biased Continuous Feedback

Unbiased Learning to Rank Meets Reality: Lessons from Baidu's Large-Scale Search Dataset

Identifiability Matters: Revealing the Hidden Recoverable Condition in Unbiased Learning to Rank

Listwise learning to rank method combining approximate NDCG ranking indicator with Conditional Generative Adversarial Networks