Global Ranking of Documents Using Continuous Conditional Random Fields

Tao Qin,Tie-Yan Liu,Xu-Dong Zhang,Hang Li
2008-01-01
Abstract:This paper is concerned with ranking model construction in document retrieval. Traditionally, the ranking model is defined as a function of a query and a document. In practice, many factors a ecting ranking can and must be taken into considera- tion, for instance, similarities between documents and hyper links between docu- ments. One needs to exploit a new ranking model which is a function of a query and the entire set of documents retrieved with the query. This paper names this new problem 'global ranking of documents', in contrast to traditional 'local rank- ing of documents'. The paper proposes a novel learning to rank method to perform the task. The method employs Continuous Conditional Random Fields (CRF) as model, which is a conditional probability distribution representing the mapping relationship from the retrieved documents to their ranking scores. The model can naturally utilize as features the content information of documents as well as the re- lation information between documents for global ranking. A learning algorithm for creating Continuous CRF is also presented in the paper. Taking Pseudo Relevance Feedback and Topic Distillation as examples, this paper shows how the learning method can be applied to global ranking. Experimental results on benchmark data show that the proposed method outperforms the baseline methods.
What problem does this paper attempt to address?