Bridging Items and Language: A Transition Paradigm for Large Language Model-Based Recommendation

Xinyu Lin,Wenjie Wang,Yongqi Li,Fuli Feng,See-Kiong Ng,Tat-Seng Chua
2024-07-25
Abstract:Harnessing Large Language Models (LLMs) for recommendation is rapidly emerging, which relies on two fundamental steps to bridge the recommendation item space and the language space: 1) item indexing utilizes identifiers to represent items in the language space, and 2) generation grounding associates LLMs' generated token sequences to in-corpus items. However, previous methods exhibit inherent limitations in the two steps. Existing ID-based identifiers (e.g., numeric IDs) and description-based identifiers (e.g., titles) either lose semantics or lack adequate distinctiveness. Moreover, prior generation grounding methods might generate invalid identifiers, thus misaligning with in-corpus items. To address these issues, we propose a novel Transition paradigm for LLM-based Recommender (named TransRec) to bridge items and language. Specifically, TransRec presents multi-facet identifiers, which simultaneously incorporate ID, title, and attribute for item indexing to pursue both distinctiveness and semantics. Additionally, we introduce a specialized data structure for TransRec to ensure generating valid identifiers only and utilize substring indexing to encourage LLMs to generate from any position of identifiers. Lastly, TransRec presents an aggregated grounding module to leverage generated multi-facet identifiers to rank in-corpus items efficiently. We instantiate TransRec on two backbone models, BART-large and LLaMA-7B. Extensive results on three real-world datasets under diverse settings validate the superiority of TransRec.
Information Retrieval
What problem does this paper attempt to address?
### The Problem the Paper Attempts to Solve This paper aims to address two core issues faced when using large language models (LLMs) for recommendations: 1. **Item Indexing**: - Existing methods can be divided into ID-based identifiers (such as numeric IDs) and description-based identifiers (such as titles). While ID-based identifiers can effectively capture the uniqueness of items, they lack explicit semantics, limiting the knowledge generalization ability of LLMs. Description-based identifiers, although rich in semantic information, lack sufficient distinctiveness due to common vocabulary and may be inconsistent with user-item interactions. 2. **Generation Grounding**: - During the generation process, existing methods may generate invalid identifiers, leading to recommendation results that are out of the corpus range. To align the generated identifiers with items in the corpus, additional matching strategies are required, which are computationally expensive. To address these issues, the authors propose a new paradigm called TransRec, which bridges the item space and language space through Multi-facet Identifiers and Position-free Constrained Generation. Specifically: - **Multi-facet Identifiers**: Each item is assigned a multi-facet identifier, including ID, title, and attributes, to pursue both distinctiveness and semantics. - **Position-free Constrained Generation**: Introducing the FM-index data structure, which supports generating valid identifiers from any position, thus avoiding the problem of generating invalid identifiers. With these improvements, TransRec demonstrates superior performance over traditional recommendation systems and LLM-based recommendation systems on multiple real-world datasets.