Large Scale Product Graph Construction for Recommendation in E-commerce

Xiaoyong Yang,Yadong Zhu,Yi Zhang,Xiaobo Wang,Quan Yuan

DOI: https://doi.org/10.48550/arXiv.2010.05525

2020-10-12

Abstract:Building a recommendation system that serves billions of users on daily basis is a challenging problem, as the system needs to make astronomical number of predictions per second based on real-time user behaviors with O(1) time complexity. Such kind of large scale recommendation systems usually rely heavily on pre-built index of products to speedup the recommendation service so that online user waiting time is un-noticeable. One important indexing structure is the product-product index, where one can retrieval a list of ranked products given a seed product. The index can be viewed as a weighted product-product graph. In this paper, we present our novel technologies to efficiently build such kind of indexed product graphs. In particular, we propose the Swing algorithm to capture the substitute relationships between products, which can utilize the substructures of user-item click bi-partitive graph. Then we propose the Surprise algorithm for the modeling of complementary product relationships, which utilizes product category information and solves the sparsity problem of user co-purchasing graph via clustering technique. Base on these two approaches, we can build the basis product graph for recommendation in Taobao. The approaches are evaluated comprehensively with both offline and online experiments, and the results demonstrate the effectiveness and efficiency of the work.

Information Retrieval

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to efficiently construct a product graph in large - scale e - commerce recommendation systems to improve the speed and accuracy of recommendation services. Specifically, the paper focuses on two main aspects:

1. **Substitution relationship**: How to effectively capture the substitution relationships between products. For example, after a user browses or purchases a shirt, the recommendation system should be able to recommend other similar shirts for the user to choose and compare.

2. **Complementary relationship**: How to effectively model the complementary relationships between products. For example, after a user purchases a mobile phone, the recommendation system should be able to recommend related accessories such as mobile phone cases and power banks.

To solve these problems, the author proposes two algorithms:

- **Swing algorithm**: It is used to capture the substitution relationships between products. This algorithm improves the accuracy and robustness of prediction by using the sub - structure of the user - item click bipartite graph, reducing the influence of noisy data.

- **Surprise algorithm**: It is used to model the complementary relationships between products. This algorithm solves the sparsity problem of the user co - purchase graph by using product category information and clustering techniques, and takes into account time - sensitivity and the order of co - purchased products.

Through these two algorithms, the paper aims to construct a basic product graph to provide efficient recommendation services for large e - commerce platforms such as Taobao, thereby enhancing user experience and commercial value.