Abstract:Recently, there has been growing interest in developing the next-generation recommender systems (RSs) based on pretrained large language models (LLMs). However, the semantic gap between natural language and recommendation tasks is still not well addressed, leading to multiple issues such as spuriously correlated user/item descriptors, ineffective language modeling on user/item data, inefficient recommendations via auto-regression, etc. In this paper, we propose CLLM4Rec, the first generative RS that tightly integrates the LLM paradigm and ID paradigm of RSs, aiming to address the above challenges simultaneously. We first extend the vocabulary of pretrained LLMs with user/item ID tokens to faithfully model user/item collaborative and content semantics. Accordingly, a novel soft+hard prompting strategy is proposed to effectively learn user/item collaborative/content token embeddings via language modeling on RS-specific corpora, where each document is split into a prompt consisting of heterogeneous soft (user/item) tokens and hard (vocab) tokens and a main text consisting of homogeneous item tokens or vocab tokens to facilitate stable and effective language modeling. In addition, a novel mutual regularization strategy is introduced to encourage CLLM4Rec to capture recommendation-related information from noisy user/item content. Finally, we propose a novel recommendation-oriented finetuning strategy for CLLM4Rec, where an item prediction head with multinomial likelihood is added to the pretrained CLLM4Rec backbone to predict hold-out items based on soft+hard prompts established from masked user-item interaction history, where recommendations of multiple items can be generated efficiently without hallucination. Codes are released at <a class="link-external link-https" href="https://github.com/yaochenzhu/llm4rec" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the semantic gap problem when using pre - trained large - scale language models (LLMs) in recommender systems (RS). Specifically, the semantic differences between natural language and recommendation tasks have not been fully addressed, leading to multiple problems, such as false correlations in user / item descriptions, poor language modeling effects on user / item data, and low efficiency in recommendation through autoregression. To solve these problems, the authors propose CLLM4Rec, which is the first generative recommender system that closely combines the LLM paradigm and the ID paradigm, aiming to simultaneously address the above challenges. The following are the specific solutions proposed in the paper: 1. **Vocabulary Expansion**: First, expand the vocabulary of the pre - trained LLM and add user / item ID tokens to faithfully model the collaborative and content semantics of users / items. 2. **Soft + Hard Prompting Strategy**: Propose a novel soft + hard prompting strategy to effectively learn the collaborative / content token embeddings of users / items through language modeling on RS - specific corpora. Each document is split into a prompt part consisting of heterogeneous soft (user / item) tokens and hard (vocabulary) tokens, and a main text part consisting of homogeneous item tokens or vocabulary tokens, in order to promote stable and effective language modeling. 3. **Reciprocal Regularization Strategy**: Introduce a new reciprocal regularization strategy to encourage CLLM4Rec to capture recommendation - related information from noisy user / item content. 4. **Recommendation - Oriented Fine - Tuning Strategy**: Propose a new recommendation - oriented fine - tuning strategy, add an item prediction head with a polynomial likelihood on the pre - trained CLLM4Rec backbone, predict the held - out items based on the soft + hard prompts established from the masked user - item interaction history, and thus efficiently generate recommendations for multiple items without hallucination. Through these methods, CLLM4Rec can better bridge the semantic gap between natural language and recommendation tasks and improve the accuracy and efficiency of the recommender system.

Collaborative Large Language Model for Recommender Systems

Collaborative Large Language Model for Recommender Systems

Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation

Towards a Unified Paradigm: Integrating Recommendation Systems as a New Language in Large Models

Large Language Models meet Collaborative Filtering: An Efficient All-round LLM-based Recommender System

Representation Learning with Large Language Models for Recommendation

CoRAL: Collaborative Retrieval-Augmented Large Language Models Improve Long-tail Recommendation

Large Language Models Enhanced Collaborative Filtering

Large Language Models for Recommendation: Progresses and Future Directions

Leveraging Large Language Models for Pre-trained Recommender Systems

Large Language Model Can Interpret Latent Space of Sequential Recommender

LLaRA: Aligning Large Language Models with Sequential Recommenders.

CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendation

Recommender Systems in the Era of Large Language Models (LLMs)

Harnessing Large Language Models for Text-Rich Sequential Recommendation

LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation

E4SRec: An Elegant Effective Efficient Extensible Solution of Large Language Models for Sequential Recommendation

Collaborative Cross-modal Fusion with Large Language Model for Recommendation