Abstract:Learning low-dimensional representation for large number of products present in an e-commerce catalogue plays a vital role as they are helpful in tasks like product ranking, product recommendation, finding similar products, modelling user-behaviour etc. Recently, a lot of tasks in the NLP field are getting tackled using the Transformer based models and these deep models are widely applicable in the industries setting to solve various problems. With this motivation, we apply transformer based model for learning contextual representation of products in an e-commerce setting. In this work, we propose a novel approach of pre-training transformer based model on a users generated sessions dataset obtained from a large fashion e-commerce platform to obtain latent product representation. Once pre-trained, we show that the low-dimension representation of the products can be obtained given the product attributes information as a textual sentence. We mainly pre-train BERT, RoBERTa, ALBERT and XLNET variants of transformer model and show a quantitative analysis of the products representation obtained from these models with respect to Next Product Recommendation(NPR) and Content Ranking(CR) tasks. For both the tasks, we collect an evaluation data from the fashion e-commerce platform and observe that XLNET model outperform other variants with a MRR of 0.5 for NPR and NDCG of 0.634 for CR. XLNET model also outperforms the Word2Vec based non-transformer baseline on both the downstream tasks. To the best of our knowledge, this is the first and novel work for pre-training transformer based models using users generated sessions data containing products that are represented with rich attributes information for adoption in e-commerce setting. These models can be further fine-tuned in order to solve various downstream tasks in e-commerce, thereby eliminating the need to train a model from scratch.

Learning Latent Vector Spaces for Product Search

Learning Exemplar-Represented Manifolds in Latent Space for Classification.

Studying Product Competition Using Representation Learning

Theoretical Understandings of Product Embedding for E-commerce Machine Learning

Learning Robust Models for e-Commerce Product Search

Learning to Rank Broad and Narrow Queries in E-Commerce

A Comparison of Supervised Learning to Match Methods for Product Search

Query2Prod2Vec Grounded Word Embeddings for eCommerce

Cluster Language Model for Improved E-Commerce Retrieval and Ranking: Leveraging Query Similarity and Fine-Tuning for Personalized Results

Towards More Relevant Product Search Ranking Via Large Language Models: An Empirical Study

ITEm: Unsupervised Image-Text Embedding Learning for eCommerce

V$^2$L: Leveraging Vision and Vision-language Models into Large-scale Product Retrieval

Learning-To-Embed: Adopting Transformer based models for E-commerce Products Representation Learning

Learning to Ask: Question-based Sequential Bayesian Product Search

Unified Vision-Language Representation Modeling for E-Commerce Same-Style Products Retrieval

Latent Space Cartography: Visual Analysis of Vector Space Embeddings

End-to-End Neural Ranking for eCommerce Product Search: an application of task models and textual embeddings

Large Scale Product Search with Spatial Quantization and Deep Ranking

Learning a Hierarchical Embedding Model for Personalized Product Search

Learning to Ask: Conversational Product Search via Representation Learning

Weighted Ensemble LSTM Model with Word Embedding Attention for E-Commerce Product Recommendation