Are GPT Embeddings Useful for Ads and Recommendation?

Wenjun Peng,Derong Xu,Tong Xu,Jianjin Zhang,Enhong Chen
DOI: https://doi.org/10.1007/978-3-031-40292-0_13
2023-01-01
Abstract:Advertisement (ads) and recommendation are important for companies to drive their business objectives and improve user loyalty. A key strategy for these services is semantic modeling, which involves extracting useful knowledge or information from text. Large language models (LLMs) such as GPT-3 and LaMDA have incredible natural language understanding capabilities and their text embeddings have achieved excellent performance in various NLP tasks. Despite their potential, the discussion about whether text embeddings of LLMs can help ads and recommendation services is limited. In order to explore the utilization of GPT embeddings for ads and recommendation, we propose three strategies to integrate LLMs’ knowledge into basic PLMs and improve their performance. These strategies consider GPT embedding as a feature (EaaF) to enrich text semantics, as a regularization (EaaR) to guide text token embedding aggregation, and as a pre-training task (EaaP) to replicate the capability of LLMs, respectively. Our experiments demonstrate that, by incorporating GPT embeddings, basic PLMs can improve their performance in both ads and recommendation tasks. Our code is available at https://github.com/Wenjun-Peng/GPT4SM
What problem does this paper attempt to address?