LLaRA: Large Language-Recommendation Assistant

Jiayi Liao,Sihang Li,Zhengyi Yang,Jiancan Wu,Yancheng Yuan,Xiang Wang,Xiangnan He

2024-05-04

Abstract:Sequential recommendation aims to predict users' next interaction with items based on their past engagement sequence. Recently, the advent of Large Language Models (LLMs) has sparked interest in leveraging them for sequential recommendation, viewing it as language modeling. Previous studies represent items within LLMs' input prompts as either ID indices or textual metadata. However, these approaches often fail to either encapsulate comprehensive world knowledge or exhibit sufficient behavioral understanding. To combine the complementary strengths of conventional recommenders in capturing behavioral patterns of users and LLMs in encoding world knowledge about items, we introduce Large Language-Recommendation Assistant (LLaRA). Specifically, it uses a novel hybrid prompting method that integrates ID-based item embeddings learned by traditional recommendation models with textual item features. Treating the "sequential behaviors of users" as a distinct modality beyond texts, we employ a projector to align the traditional recommender's ID embeddings with the LLM's input space. Moreover, rather than directly exposing the hybrid prompt to LLMs, a curriculum learning strategy is adopted to gradually ramp up training complexity. Initially, we warm up the LLM using text-only prompts, which better suit its inherent language modeling ability. Subsequently, we progressively transition to the hybrid prompts, training the model to seamlessly incorporate the behavioral knowledge from the traditional sequential recommender into the LLM. Empirical results validate the effectiveness of our proposed framework. Codes are available at <a class="link-external link-https" href="https://github.com/ljy0ustc/LLaRA" rel="external noopener nofollow">this https URL</a>.

Information Retrieval

What problem does this paper attempt to address?

The paper aims to address the next item prediction problem in sequential recommendation. Specifically, it proposes a new framework—LLaRA (Large Language-Recommendation Assistant), which combines the behavioral patterns of traditional sequential recommendation models with the world knowledge of large language models (LLMs). Traditional recommendation systems capture behavioral patterns through users' historical interaction data, while LLMs excel at encoding extensive world knowledge about items. However, directly inputting this information to LLMs in the form of IDs or text often fails to fully utilize their potential.

The main contributions of the paper are:
1. **Hybrid Prompt Design**: A novel hybrid prompt method is introduced, which combines ID-based item embeddings learned from traditional recommendation models with textual features of items to form a multifaceted item representation. Specifically, a projector is used to convert traditional ID-based item embeddings into behavior tokens compatible with the LLMs' text space, and these are combined with text tokens.
2. **Curriculum Prompt Tuning Strategy**: A curriculum learning approach is adopted to gradually transition from pure text prompts to hybrid prompts, allowing LLMs to first become familiar with the recommendation mechanism through pure text prompts and then gradually internalize the behavioral knowledge from traditional recommendation models through hybrid prompts. This method ensures that LLMs' performance in sequential recommendation tasks is enhanced, particularly in understanding user behavior.

In this way, LLaRA not only leverages the powerful language processing capabilities of LLMs but also retains the advantages of traditional recommendation systems, thereby demonstrating superior performance compared to other baseline models in experiments.