Abstract:As the heart of a search engine, the ranking system plays a crucial role in satisfying users' information demands. More recently, neural rankers fine-tuned from pre-trained language models (PLMs) establish state-of-the-art ranking effectiveness. However, it is nontrivial to directly apply these PLM-based rankers to the large-scale web search system due to the following challenging issues: (1) the prohibitively expensive computations of massive neural PLMs, especially for long texts in the web document, prohibit their deployments in an online ranking system that demands extremely low latency; (2) the discrepancy between existing ranking-agnostic pre-training objectives and the ad-hoc retrieval scenarios that demand comprehensive relevance modeling is another main barrier for improving the online ranking system; (3) a real-world search engine typically involves a committee of ranking components, and thus the compatibility of the individually fine-tuned ranking model is critical for a cooperative ranking system. In this work, we contribute a series of successfully applied techniques in tackling these exposed issues when deploying the state-of-the-art Chinese pre-trained language model, i.e., ERNIE, in the online search engine system. We first articulate a novel practice to cost-efficiently summarize the web document and contextualize the resultant summary content with the query using a cheap yet powerful Pyramid-ERNIE architecture. Then we endow an innovative paradigm to finely exploit the large-scale noisy and biased post-click behavioral data for relevance-oriented pre-training. We also propose a human-anchored fine-tuning strategy tailored for the online ranking system, aiming to stabilize the ranking signals across various online components. Extensive offline and online experimental results show that the proposed techniques significantly boost the search engine's performance.

Multi Page Search with Reinforcement Learning to Rank

Reinforcement Learning to Rank with Markov Decision Process

Ranking with Deep Multi-Objective Learning

RLPS: A Reinforcement Learning–Based Framework for Personalized Search

Improving Session Search Performance with a Multi-MDP Model.

Context-Aware Ranking by Constructing a Virtual Environment for Reinforcement Learning

RLPer: A Reinforcement Learning Model for Personalized Search.

Adapting Markov Decision Process for Search Result Diversification

Multi-task Learning to Rank for Web Search

A Multi-Perspective Learning to Rank Approach to Support Children's Information Seeking in the Classroom

Reinforcement Learning to Rank with Pairwise Policy Gradient

Cooperative Multi-Agent Deep Reinforcement Learning in Content Ranking Optimization

Learning to Collaborate: Multi-Scenario Ranking Via Multi-Agent Reinforcement Learning.

Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximization

Pre-trained Language Model based Ranking in Baidu Search

Sequential Search with Off-Policy Reinforcement Learning

GS2P: a generative pre-trained learning to rank model with over-parameterization for web-scale search

Learning to Re-Rank Web Search Results with Multiple Pairwise Features

Integrated Ranking for News Feed with Reinforcement Learning

Multi-Scenario Ranking with Adaptive Feature Learning

Pretrained Language Model based Web Search Ranking: From Relevance to Satisfaction