Abstract:Effective query-item relevance modeling is pivotal for enhancing user experience and safeguarding user satisfaction in e-commerce search systems. Recently, benefiting from the vast inherent knowledge, Large Language Model (LLM) approach demonstrates strong performance and long-tail generalization ability compared with previous neural-based specialized relevance learning methods. Though promising, current LLM-based methods encounter the following inadequacies in practice: First, the massive parameters and computational demands make it difficult to be deployed online. Second, distilling LLM models to online models is a feasible direction, but the LLM relevance modeling is a black box, and its rich intrinsic knowledge is difficult to extract and apply online. To improve the interpretability of LLM and boost the performance of online relevance models via LLM, we propose an Explainable LLM-driven Multi-dimensional Distillation framework for e-commerce relevance learning, which comprises two core components: (1) An Explainable LLM for relevance modeling (ELLM-rele), which decomposes the relevance learning into intermediate steps and models relevance learning as a Chain-of-Thought (CoT) reasoning, thereby enhancing both interpretability and performance of LLM. (2) A Multi-dimensional Knowledge Distillation (MKD) architecture that transfers the knowledge of ELLM-rele to current deployable interaction-based and representation-based student models from both the relevance score distribution and CoT reasoning aspects. Through distilling the probabilistic and CoT reasoning knowledge, MKD improves both the semantic interaction and long-tail generalization abilities of student models. Extensive offline evaluations and online experiments on Taobao search ad scene demonstrate that our proposed framework significantly enhances e-commerce relevance learning performance and user experience.

EC-Guide: A Comprehensive E-Commerce Guide for Instruction Tuning and Quantization

LLaSA: Large Language and E-Commerce Shopping Assistant

eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data

Winning Amazon KDD Cup'24

EcomGPT: Instruction-tuning Large Language Models with Chain-of-Task Tasks for E-commerce

LLaMA-E: Empowering E-commerce Authoring with Object-Interleaved Instruction Following

Investigating LLM Applications in E-Commerce

Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Models

Llms-Based Machine Translation for E-Commerce

ZhichunRoad at Amazon KDD Cup 2022: MultiTask Pre-Training for E-Commerce Product Search

EcomEdit: An Automated E-commerce Knowledge Editing Framework for Enhanced Product and Purchase Intention Understanding

General2Specialized LLMs Translation for E-commerce

Some Practice for Improving the Search Results of E-commerce

A Boring-yet-effective Approach for the Product Ranking Task of the Amazon KDD Cup 2022

A Semantic Alignment System for Multilingual Query-Product Retrieval

Large Language Models for Supply Chain Optimization

Explainable LLM-driven Multi-dimensional Distillation for E-Commerce Relevance Learning

EcomGPT-CT: Continual Pre-training of E-commerce Large Language Models with Semi-structured Data

Beyond the Ground-Truth: an Evaluator-Generator Framework for Group-wise Learning-to-Rank in E-Commerce

LLM-Ensemble: Optimal Large Language Model Ensemble Method for E-commerce Product Attribute Value Extraction

CommonIT: Commonality-Aware Instruction Tuning for Large Language Models via Data Partitions