GEMRec: Towards Generative Model Recommendation

Yuanhe Guo,Haoming Liu,Hongyi Wen
DOI: https://doi.org/10.1145/3616855.3635700
2023-12-07
Abstract:Recommender Systems are built to retrieve relevant items to satisfy users' information needs. The candidate corpus usually consists of a finite set of items that are ready to be served, such as videos, products, or articles. With recent advances in Generative AI such as GPT and Diffusion models, a new form of recommendation task is yet to be explored where items are to be created by generative models with personalized prompts. Taking image generation as an example, with a single prompt from the user and access to a generative model, it is possible to generate hundreds of new images in a few minutes. How shall we attain personalization in the presence of "infinite" items? In this preliminary study, we propose a two-stage framework, namely Prompt-Model Retrieval and Generated Item Ranking, to approach this new task formulation. We release GEMRec-18K, a prompt-model interaction dataset with 18K images generated by 200 publicly-available generative models paired with a diverse set of 90 textual prompts. Our findings demonstrate the promise of generative model recommendation as a novel personalization problem and the limitations of existing evaluation metrics. We highlight future directions for the RecSys community to advance towards generative recommender systems. Our code and dataset are available at <a class="link-external link-https" href="https://github.com/MAPS-research/GEMRec" rel="external noopener nofollow">this https URL</a>.
Information Retrieval,Artificial Intelligence
What problem does this paper attempt to address?
The paper aims to explore a new recommendation system task—Generative Model Recommendation. Specifically, the paper focuses on the problem of achieving personalized recommendations within generative models (such as text-to-image models). Traditional recommendation systems primarily retrieve the most relevant items from a finite set of items to meet user needs. However, with the development of generative models, it is possible to generate an almost infinite number of new items. Therefore, how to achieve personalized recommendations in this "infinite" item space has become a new challenge. To address this challenge, the authors propose a two-stage framework: 1. **Prompt-Model Retrieval**: First, based on the user's prompt, a set of models related to the prompt is retrieved from a large number of generative models. 2. **Generative Model Ranking**: In the smaller candidate model set, the models are ranked based on user feedback to select the model that best fits the user's preferences. Additionally, the paper releases a dataset named GEMRec-18K, which contains 18,000 images generated by 200 publicly available generative models and 90 text prompts. This dataset is used to explore generative model recommendations and can be used to understand the characteristics of different generative models. The paper also proposes a Generative Recommendation Evaluation Score (GRE-Score) to pre-rank the retrieved candidate images and their associated models. Finally, the authors developed an interactive interface to demonstrate the workflow of the entire framework and discussed future research directions.