Abstract:The conventional solution to learning to rank problems ranks individual documents by prediction scores greedily. Recent emerged re-ranking models, which take as input initial lists, aim to capture document interdependencies and directly generate the optimal ordered lists. Typically, a re-ranking model is learned from a set of labeled data, which can achieve favorable performance on average. However, it can be suboptimal for individual queries because the available training data is usually highly imbalanced. This problem is challenging due to the absence of informative data for some queries and furthermore, the lack of a good data augmentation policy. In this paper, we propose a novel method named Learning to Augment (LTA), which mitigates the imbalance issue through learning to augment the initial lists for re-ranking models. Specifically, we first design a data generation model based on Gaussian Mixture Variational Autoencoder (GMVAE) for generating informative data. GMVAE imposes a mixture of Gaussians on the latent space, which allows it to cluster queries in an unsupervised manner and then generate new data with different query types using the learned components. Then, to obtain a good augmentation strategy (instead of heuristics), we design a teacher model that consists of two intelligent agents to determine how to generate new data for a given list and how to rank both the raw data and generated data to produce augmented lists, respectively. The teacher model leverages the feedback from the re-ranking model to optimize its augmentation policy by means of reinforcement learning. Our method offers a general learning paradigm that is applicable to both supervised and reinforced re-ranking models. Experimental results on benchmark learning to rank datasets show that our proposed method can significantly improve the performance of re-ranking models.

Cost-Sensitive Supported Vector Learning To Rank Imbalanced Data Set

Cost Sensitive Ranking Support Vector Machine for Multi-label Data Learning.

Self-adaptive cost weights-based support vector machine cost-sensitive ensemble for imbalanced data classification

Online Learning to Rank in a Listwise Approach for Information Retrieval

Optimizing Cost-Sensitive SVM for Imbalanced Data : Connecting Cluster to Classification.

Scalar is Not Enough: Vectorization-based Unbiased Learning to Rank

Cost Sensitive Support Vector Machines

Learning to rank with minimal set of hyperplanes for document retrieval

Cost-Sensitive Learning based on Performance Metric for Imbalanced Data

RUE: A Robust Personalized Cost Assignment Strategy for Class Imbalance Cost-sensitive Learning

Optimization of Selective Ensemble for Cost-Sensitive Classification: an Empirical Study

A Cost-Sensitive Multi-Criteria Quadratic Programming Model for Imbalanced Data

SetRank: Learning a Permutation-Invariant Ranking Model for Information Retrieval

A New Svm-Rfe Approach Towards Ranking Problem

Learning to Rank Relational Objects Based on the Listwise Approach

A Biased Sampling Method for Imbalanced Personalized Ranking

Learning to Augment Imbalanced Data for Re-ranking Models.

Selecting optimal training data for learning to rank

Learning to rank with voted multiple hyperplanes for documents retrieval

Ranking Estimation Performance by Estimator Randomization and Attribute Support