Learnable Item Tokenization for Generative Recommendation

Wenjie Wang,Honghui Bao,Xinyu Lin,Jizhi Zhang,Yongqi Li,Fuli Feng,See-Kiong Ng,Tat-Seng Chua

2024-08-19

Abstract:Utilizing powerful Large Language Models (LLMs) for generative recommendation has attracted much attention. Nevertheless, a crucial challenge is transforming recommendation data into the language space of LLMs through effective item tokenization. Current approaches, such as ID, textual, and codebook-based identifiers, exhibit shortcomings in encoding semantic information, incorporating collaborative signals, or handling code assignment bias. To address these limitations, we propose LETTER (a LEarnable Tokenizer for generaTivE Recommendation), which integrates hierarchical semantics, collaborative signals, and code assignment diversity to satisfy the essential requirements of identifiers. LETTER incorporates Residual Quantized VAE for semantic regularization, a contrastive alignment loss for collaborative regularization, and a diversity loss to mitigate code assignment bias. We instantiate LETTER on two models and propose a ranking-guided generation loss to augment their ranking ability theoretically. Experiments on three datasets validate the superiority of LETTER, advancing the state-of-the-art in the field of LLM-based generative recommendation.

Information Retrieval

What problem does this paper attempt to address?

The paper aims to address the key challenge of item tokenization encountered when using large language models (LLMs) for item encoding and generation in generative recommendation systems. Specifically:

1. **The Need for Item Tokenization**: Transforming recommendation data into a language space that LLMs can process, i.e., mapping each item into a series of identifiers (token sequences) through effective item tokenization methods to connect recommendation data with the way LLMs operate.

2. **Limitations of Existing Methods**:
- **ID Identifiers**: While ensuring uniqueness, they lack semantic information and are difficult to extend to cold-start items.
- **Text Identifiers**: Directly using information such as item titles as identifiers leads to a loss of hierarchical semantics and the absence of collaborative signals from user behavior.
- **Dictionary-Based Methods**: Encoding item semantics into hierarchical code sequences through autoencoders, but similarly lacking collaborative signals and facing generation bias due to uneven code distribution.

3. **Goals**: Ideal identifiers should have the following characteristics:
- Hierarchical Semantic Structure: Gradually encoding the semantic information of items from coarse-grained to fine-grained.
- Integration of Collaborative Signals: Ensuring that items with similar collaborative patterns have similar identifier sequences.
- Enhanced Code Distribution Diversity: Reducing code allocation bias to ensure fairness in item generation.

To this end, the authors propose a method named LETTER (a Learnable Tokenizer for generative recommendation), which optimizes dictionary-based identifiers by introducing three types of regularization: semantic regularization, collaborative regularization, and diversity regularization, thereby improving the quality and effectiveness of item tokenization in multiple aspects.