RecExplainer: Aligning Large Language Models for Explaining Recommendation Models

Yuxuan Lei,Jianxun Lian,Jing Yao,Xu Huang,Defu Lian,Xing Xie

DOI: https://doi.org/10.1145/3637528.3671802

2024-06-22

Abstract:Recommender systems are widely used in online services, with embedding-based models being particularly popular due to their expressiveness in representing complex signals. However, these models often function as a black box, making them less transparent and reliable for both users and developers. Recently, large language models (LLMs) have demonstrated remarkable intelligence in understanding, reasoning, and instruction following. This paper presents the initial exploration of using LLMs as surrogate models to explaining black-box recommender models. The primary concept involves training LLMs to comprehend and emulate the behavior of target recommender models. By leveraging LLMs' own extensive world knowledge and multi-step reasoning abilities, these aligned LLMs can serve as advanced surrogates, capable of reasoning about observations. Moreover, employing natural language as an interface allows for the creation of customizable explanations that can be adapted to individual user preferences. To facilitate an effective alignment, we introduce three methods: behavior alignment, intention alignment, and hybrid alignment. Behavior alignment operates in the language space, representing user preferences and item information as text to mimic the target model's behavior; intention alignment works in the latent space of the recommendation model, using user and item representations to understand the model's behavior; hybrid alignment combines both language and latent spaces. Comprehensive experiments conducted on three public datasets show that our approach yields promising results in understanding and mimicking target models, producing high-quality, high-fidelity, and distinct explanations. Our code is available at <a class="link-external link-https" href="https://github.com/microsoft/RecAI" rel="external noopener nofollow">this https URL</a>.

Information Retrieval,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the interpretability problem of models in recommendation systems. Specifically, recommendation systems are widely used in various online services. In particular, embedding - based recommendation models are especially popular due to their expressiveness in representing complex signals. However, these models usually operate in a black - box manner, resulting in insufficient transparency and reliability, which not only affects user trust but also poses challenges to developers.

The main contribution of the paper lies in exploring the possibility of using large - language models (LLMs) as proxy models to explain black - box recommendation models. By training LLMs to understand and imitate the behavior of the target recommendation model, and by leveraging the rich world knowledge and multi - step reasoning ability of LLMs themselves, these aligned LLMs can serve as high - level proxies and be able to reason about observations. In addition, using natural language as an interface allows for the creation of customizable explanations that can be adjusted according to the preferences of individual users.

To achieve effective alignment, the paper introduces three methods: behavior alignment, intent alignment, and hybrid alignment. These methods work at different levels respectively, aiming to improve the understanding and interpretive ability of LLMs for recommendation models. Through comprehensive experiments on three public datasets, the research has proven that this method has achieved encouraging results in understanding and imitating the target model and can generate high - quality, high - fidelity, and distinctive explanations.

In summary, the goal of this paper is to improve the interpretability of recommendation systems by using LLMs as proxy models, thereby enhancing the reliability of the system and user trust.