Algorithmic Drift: A Simulation Framework to Study the Effects of Recommender Systems on User Preferences

Erica Coppolillo,Simone Mungari,Ettore Ritacco,Francesco Fabbri,Marco Minici,Francesco Bonchi,Giuseppe Manco

2024-09-25

Abstract:Digital platforms such as social media and e-commerce websites adopt Recommender Systems to provide value to the user. However, the social consequences deriving from their adoption are still unclear. Many scholars argue that recommenders may lead to detrimental effects, such as bias-amplification deriving from the feedback loop between algorithmic suggestions and users' choices. Nonetheless, the extent to which recommenders influence changes in users leaning remains uncertain. In this context, it is important to provide a controlled environment for evaluating the recommendation algorithm before deployment. To address this, we propose a stochastic simulation framework that mimics user-recommender system interactions in a long-term scenario. In particular, we simulate the user choices by formalizing a user model, which comprises behavioral aspects, such as the user resistance towards the recommendation algorithm and their inertia in relying on the received suggestions. Additionally, we introduce two novel metrics for quantifying the algorithm's impact on user preferences, specifically in terms of drift over time. We conduct an extensive evaluation on multiple synthetic datasets, aiming at testing the robustness of our framework when considering different scenarios and hyper-parameters setting. The experimental results prove that the proposed methodology is effective in detecting and quantifying the drift over the users preferences by means of the simulation. All the code and data used to perform the experiments are publicly available.

Information Retrieval,Artificial Intelligence,Social and Information Networks

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

The paper primarily focuses on the long-term impact of recommender systems on user preferences and proposes a simulation framework to study this issue. Specifically, the paper attempts to address the following problems:

1. **Impact of Recommendation Algorithms on User Preference Changes**:
- The paper explores how recommendation algorithms change user preferences over time. While existing research has shown that recommender systems may lead to negative phenomena such as echo chambers and opinion polarization, these studies mostly focus on specific aspects and lack a unified approach to track the evolution of user preferences.

2. **Quantifying Algorithmic Drift in Recommender Systems**:
- The paper defines the concept of "Algorithmic Drift" and introduces two new metrics to quantify the long-term impact of recommender systems on user behavior changes. By simulating the interaction process between users and recommender systems, the paper aims to measure whether and to what extent recommendation algorithms lead to changes in user preferences.

3. **Impact of Different User Behavior Patterns**:
- The paper also investigates how different user behavior patterns (such as resistance, inertia, randomness) affect the effectiveness of recommender systems. For example, users may completely ignore the recommendations, rely on their own interests to choose content, or randomly select content.

Through the above research, the paper aims to provide a controlled simulation environment to test the impact of recommendation algorithms before actual deployment. The paper proposes a simulation-based framework and introduces two new metrics (Algorithmic Drift Score and Delta Target Consumption) to evaluate the long-term effects of recommender systems.