Abstract:The search engine plays a fundamental role in online e-commerce systems, to help users find the products they want from the massive product collections. Relevance is an essential requirement for e-commerce search, since showing products that do not match search query intent will degrade user experience. With the existence of vocabulary gap between user language of queries and seller language of products, measuring semantic relevance is necessary and neural networks are engaged to address this task. However, semantic relevance is different from click-through rate prediction in that no direct training signal is available. Most previous attempts learn relevance models from user click-through data that are cheap and abundant. Unfortunately, click behavior is noisy and misleading, which is affected by not only relevance but also factors including price, image and attractive titles. Therefore, it is challenging but valuable to learn relevance models from click-through data. In this paper, we propose a new relevance learning framework that concentrates on how to train a relevance model from the weak supervision of click-through data. Different from previous efforts that treat samples as either relevant or irrelevant, we construct more fine-grained samples for training. We propose a novel way to consider samples of different relevance confidence, and come up with a new training objective to learn a robust relevance model with desirable score distribution. The proposed model is evaluated on offline annotated data and online A/B testing, and it achieves both promising performance and high computational efficiency. The model has already been deployed online, serving the search traffic of Taobao for over a year.

Generating True Relevance Labels in Chinese Search Engine Using Clickthrough Data

A Mouse-Trajectory Based Model for Pre Dicting Query-Url Relevance

Exploring relevance for clicks.

Are click-through data adequate for learning web search rankings?

Ads-clicking Intent Analysis of Search Engine Queries

Learning a Product Relevance Model from Click-Through Data in E-Commerce

Characterizing Expertise of Search Engine Users.

Training Deep Ranking Model with Weak Relevance Labels

Explore click models for search ranking.

Improving Click Model by Combining Mouse Movements with Click-Through Data

Estimating Credibility of User Clicks with Mouse Movement and Eye-Tracking Information.

Automatically generating labels based on unified click model.

A Noise-Aware Click Model for Web Search

Click-Through-Based Cross-View Learning For Image Search

When does Relevance Mean Usefulness and User Satisfaction in Web Search?

Why Don't You Click: Understanding Non-Click Results inWeb Search with Brain Signals

Modeling Perceived Relevance for Tail Queries without Click-Through Data

Efficiently collecting relevance information from clickthroughs for web retrieval system evaluation.

Automatic Search Engine Performance Evaluation with Click-Through Data Analysis

Individual Judgments Versus Consensus: Estimating Query-URL Relevance.

Joint relevance and freshness learning from clickthroughs for news search.