Abstract:Sequential recommendation is to predict the next item of interest for a user, based on her/his interaction history with previous items. In conventional sequential recommenders, a common approach is to model item sequences using discrete IDs, learning representations that encode sequential behaviors and reflect user preferences. Inspired by recent success in empowering large language models (LLMs) to understand and reason over diverse modality data (e.g., image, audio, 3D points), a compelling research question arises: ``Can LLMs understand and work with hidden representations from ID-based sequential recommenders?''.To answer this, we propose a simple framework, RecInterpreter, which examines the capacity of open-source LLMs to decipher the representation space of sequential recommenders. Specifically, with the multimodal pairs (\ie representations of interaction sequence and text narrations), RecInterpreter first uses a lightweight adapter to map the representations into the token embedding space of the LLM. Subsequently, it constructs a sequence-recovery prompt that encourages the LLM to generate textual descriptions for items within the interaction sequence. Taking a step further, we propose a sequence-residual prompt instead, which guides the LLM in identifying the residual item by contrasting the representations before and after integrating this residual into the existing sequence. Empirical results showcase that our RecInterpreter enhances the exemplar LLM, LLaMA, to understand hidden representations from ID-based sequential recommenders, especially when guided by our sequence-residual prompts. Furthermore, RecInterpreter enables LLaMA to instantiate the oracle items generated by generative recommenders like DreamRec, concreting the item a user would ideally like to interact with next. Codes are available at <a class="link-external link-https" href="https://github.com/YangZhengyi98/RecInterpreter" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is: **Can large - language models (LLMs) understand and interpret the hidden representation space of ID - based sequential recommendation systems?** ### Specific problem background: 1. **Sequential recommendation systems**: Such systems aim to predict the next item that a user may be interested in based on the user's interaction history with previous items. Traditional sequential recommendation systems usually use discrete IDs to model item sequences and learn representations that can encode sequential behaviors and reflect user preferences. 2. **Large - language models (LLMs)**: With the success of LLMs in understanding and processing multi - modal data (such as images, audio, 3D point clouds, etc.), an interesting research question is: Can LLMs understand and process the hidden representations from ID - based sequential recommendation systems? ### The method proposed in the paper: To answer the above questions, the author proposes a simple framework - **RecInterpreter**, which aims to evaluate the ability of open - source LLMs (such as LLaMA) to decode the representation space of sequential recommendation systems. The specific methods are as follows: 1. **Light - weight adapter**: Map the representation of the recommendation system to the word - embedding space of the LLM through a light - weight adapter. 2. **Sequence - recovery prompt**: Construct a text prompt to encourage the LLM to generate text descriptions of items in the interaction sequence. 3. **Sequence - residual prompt**: Design a more refined prompt to guide the LLM to identify residual items by comparing the representations before and after integrating new items in the sequence. ### Main contributions: 1. **Verified the understanding ability of LLMs**: Empirical results show that RecInterpreter can enhance LLaMA's understanding of the hidden representations of ID - based sequential recommendation systems, especially when using the sequence - residual prompt. 2. **Generate explicit decoding of recommendation systems**: RecInterpreter can also enable LLaMA to generate specific descriptions of items generated by generative recommendation systems (such as DreamRec), thereby completing the explicit decoding of recommendation tasks. ### Experimental results: - **Sequence - recovery task**: Experimental results show that LLaMA can recover user - interacted items from hidden representations to a certain extent, but it is more difficult to fully recover all items. - **Sequence - residual task**: By comparing the representations before and after integrating new items in the sequence, LLaMA can identify new items more accurately, showing stronger understanding ability. ### Conclusion: This research proves that LLMs can understand the hidden representations of sequential recommendation systems through appropriate prompts and adapters, and can generate specific item descriptions in generative recommendation systems, providing a new way for the interpretability and transparency of recommendation systems.

Large Language Model Can Interpret Latent Space of Sequential Recommender

LLaRA: Aligning Large Language Models with Sequential Recommenders.

LLaRA: Large Language-Recommendation Assistant

Towards a Unified Paradigm: Integrating Recommendation Systems as a New Language in Large Models

Representation Learning with Large Language Models for Recommendation

Improving Sequential Recommendations with LLMs

Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation

XRec: Large Language Models for Explainable Recommendation

Rethinking Large Language Model Architectures for Sequential Recommendations

SLMRec: Empowering Small Language Models for Sequential Recommendation

Collaborative Large Language Model for Recommender Systems

LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation

RecExplainer: Aligning Large Language Models for Explaining Recommendation Models

ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation

LLMEmb: Large Language Model Can Be a Good Embedding Generator for Sequential Recommendation

Large Language Models for Recommendation: Past, Present, and Future

Large Language Models for Recommendation: Progresses and Future Directions

GenRec: Large Language Model for Generative Recommendation

E4SRec: An Elegant Effective Efficient Extensible Solution of Large Language Models for Sequential Recommendation

Enhancing Sequential Recommenders with Augmented Knowledge from Aligned Large Language Models

Can Small Language Models be Good Reasoners for Sequential Recommendation?