Abstract:Recent state-of-the-art recommender systems predominantly rely on either implicit or explicit feedback from users to suggest new items. While effective in recommending novel options, many recommender systems often use uninterpretable embeddings to represent user preferences. This lack of transparency not only limits user understanding of why certain items are suggested but also reduces the user's ability to scrutinize and modify their preferences, thereby affecting their ability to receive a list of preferred recommendations. Given the recent advances in Large Language Models (LLMs), we investigate how a properly crafted prompt can be used to summarize a user's preferences from past reviews and recommend items based only on language-based preferences. In particular, we study how LLMs can be prompted to generate a natural language (NL) user profile that holistically describe a user's preferences. These NL profiles can then be leveraged to fine-tune a LLM using only NL profiles to make transparent and scrutable recommendations. Furthermore, we validate the scrutability of our user profile-based recommender by investigating the impact on recommendation changes after editing NL user profiles. According to our evaluations of the model's rating prediction performance on two benchmarking rating prediction datasets, we observe that this novel approach maintains a performance level on par with established recommender systems in a warm-start setting. With a systematic analysis into the effect of updating user profiles and system prompts, we show the advantage of our approach in easier adjustment of user preferences and a greater autonomy over users' received recommendations.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to address the issues of transparency and auditability in recommendation systems. Current recommendation systems primarily rely on users' implicit or explicit feedback to recommend new items. Although effective in recommending novel options, these systems often use inexplicable embedding vectors to represent user preferences. This lack of transparency not only limits users' understanding of the reasons behind recommendations but also reduces their ability to review and modify their preferences, thereby affecting their ability to receive personalized recommendations. To address these issues, the authors propose a method based on natural language user profiles, generating natural language descriptions of user preferences through large language models (LLMs) and using these natural language profiles for transparent and auditable recommendations. Specifically, the authors investigate how to summarize user preferences from past reviews through appropriate prompts and recommend items based solely on language preferences. Additionally, the authors validate the impact of editing natural language user profiles on recommendation results to demonstrate the advantages of this method in adjusting user preferences. ### Main Contributions 1. **Generating Natural Language User Preference Profiles**: Using large language models fine-tuned with prompt instructions to generate fluent, informative, concise, and relevant natural language user preference profiles. 2. **Recommendation Techniques Based on Natural Language Input**: Prioritizing natural language input over learned embedding vectors while maintaining performance comparable to non-transparent baseline recommendation systems. 3. **Analyzing the Impact of Editing User Profiles on Recommendation Tasks**: Investigating how editing user profiles affects the performance of downstream recommendation tasks. ### Method Overview 1. **User Preference Identification and Ranking**: By analyzing feature words and ratings in user reviews, calculating the average rating and utility score for each feature to identify and rank user preferences. 2. **Generating Natural Language Profiles**: Using large language models fine-tuned with prompt instructions to generate natural language profiles based on user reviews, which comprehensively describe user preferences. 3. **Recommendation Task**: Using the generated natural language profiles to train the recommendation system, predicting user ratings for target items through a regression task. ### Experimental Setup - **Datasets**: Using two publicly available interpretable recommendation datasets, Amazon Movies and TV and TripAdvisor. - **Evaluation Metrics**: Using common metrics such as RMSE, MAE, MAP, and nDCG to evaluate recommendation accuracy. - **Baseline Models**: Comparing with multiple baseline recommendation systems, including Most Popular, UserKNN-BM25, Item-KNN-cosine, MF, NeuMF, etc. ### Results and Analysis - **Recommendation Performance**: Experimental results show that UPR performs well on all evaluation metrics and is competitive compared to baseline models. - **Impact of Feature Quantity on Recommendation Performance**: Ablation experiments reveal that increasing the number of features can reduce loss, but performance improvement diminishes as the number of features increases. - **User Study**: Evaluating the quality of generated natural language profiles through user studies, results show that these profiles perform well in terms of fluency, informativeness, and relevance, but slightly less so in conciseness. ### Conclusion The proposed recommendation method based on natural language user profiles (UPR) improves the transparency and auditability of the recommendation process while maintaining performance comparable to existing recommendation systems. This method enables users to more easily understand and adjust their preferences, resulting in recommendations that better meet their needs. Future research will further explore human interaction with UPR and the trade-offs between transparent, auditable recommendation systems and traditional recommendation systems.

Transparent and Scrutable Recommendations Using Natural Language User Profiles

A Prompting-Based Representation Learning Method for Recommendation with Large Language Models

PALR: Personalization Aware LLMs for Recommendation

Personalized Recommendation Systems Powered By Large Language Models: Integrating Semantic Understanding and User Preferences

End-to-end Training for Recommendation with Language-based User Profiles

Language-Based User Profiles for Recommendation

A Survey on Large Language Models for Personalized and Explainable Recommendations

Empowering Few-Shot Recommender Systems with Large Language Models -- Enhanced Representations

Empowering Few-Shot Recommender Systems With Large Language Models-Enhanced Representations

HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modeling

Recommender Systems in the Era of Large Language Models (LLMs)

Mitigating Propensity Bias of Large Language Models for Recommender Systems

LLMRG: Improving Recommendations through Large Language Model Reasoning Graphs

Representation Learning with Large Language Models for Recommendation

LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking

Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction

Uncertainty-Aware Explainable Recommendation with Large Language Models

LLaRA: Aligning Large Language Models with Sequential Recommenders.

Reinforced Prompt Personalization for Recommendation with Large Language Models

Large Language Model Driven Recommendation

Let Me Do It For You: Towards LLM Empowered Recommendation via Tool Learning