Learning to Retrieve for Job Matching
Jianqiang Shen,Yuchin Juan,Shaobo Zhang,Ping Liu,Wen Pu,Sriram Vasudevan,Qingquan Song,Fedor Borisyuk,Kay Qianqi Shen,Haichao Wei,Yunxiang Ren,Yeou S. Chiou,Sicong Kuang,Yuan Yin,Ben Zheng,Muchen Wu,Shaghayegh Gharghabi,Xiaoqing Wang,Huichao Xue,Qi Guo,Daniel Hewlett,Luke Simon,Liangjie Hong,Wenjing Zhang
DOI: https://doi.org/10.48550/arXiv.2402.13435
2024-02-21
Abstract:Web-scale search systems typically tackle the scalability challenge with a two-step paradigm: retrieval and ranking. The retrieval step, also known as candidate selection, often involves extracting standardized entities, creating an inverted index, and performing term matching for retrieval. Such traditional methods require manual and time-consuming development of query models. In this paper, we discuss applying learning-to-retrieve technology to enhance LinkedIns job search and recommendation systems. In the realm of promoted jobs, the key objective is to improve the quality of applicants, thereby delivering value to recruiter customers. To achieve this, we leverage confirmed hire data to construct a graph that evaluates a seeker's qualification for a job, and utilize learned links for retrieval. Our learned model is easy to explain, debug, and adjust. On the other hand, the focus for organic jobs is to optimize seeker engagement. We accomplished this by training embeddings for personalized retrieval, fortified by a set of rules derived from the categorization of member feedback. In addition to a solution based on a conventional inverted index, we developed an on-GPU solution capable of supporting both KNN and term matching efficiently.
Information Retrieval,Machine Learning