A Flexible Large-Scale Similar Product Identification System in E-Commerce

Zhen Zuo,Lixi Wang,Michinari Momma,Wenbo Wang,Yikai Ni,Jian-su Lin,Yi Sun
2020-01-01
Abstract:Identifying similar products is a common pain-point in the world of E-commerce search and discovery.The key challenges lie in two aspects: 1) The definition of similarity varies across different applications, such as near identical products sold by different vendors, products that are substitutable to each other for customers with common interests, personalized products visually similar in terms of design pattern, style or color. It is difficult to build a general solution for all scenarios. 2) Computing pairwise similarities among billions of products are resource demanding which makes it challenging in large-scale data processing. To provide a flexible and consolidated solution at scale, this paper presents an all-in-one system, Product Similarity Service (PSS), which leverages the stateof-the-art Deep Neural Networks and distributed computing technology to serve diverse Amazon-scale product similarity computations. The experimental results show that PSS can return highly relevant similar products in both verification and ranking tasks, and has good computing efficiency and system scalability on large data volume.
What problem does this paper attempt to address?