Abstract:Recommender systems are expected to be assistants that help human users find relevant information automatically without explicit queries. As recommender systems evolve, increasingly sophisticated learning techniques are applied and have achieved better performance in terms of user engagement metrics such as clicks and browsing time. The increase in the measured performance, however, can have two possible attributions: a better understanding of user preferences, and a more proactive ability to utilize human bounded rationality to seduce user over-consumption. A natural following question is whether current recommendation algorithms are manipulating user preferences. If so, can we measure the manipulation level? In this paper, we present a general framework for benchmarking the degree of manipulations of recommendation algorithms, in both slate recommendation and sequential recommendation scenarios. The framework consists of four stages, initial preference calculation, training data collection, algorithm training and interaction, and metrics calculation that involves two proposed metrics. We benchmark some representative recommendation algorithms in both synthetic and real-world datasets under the proposed framework. We have observed that a high online click-through rate does not necessarily mean a better understanding of user initial preference, but ends in prompting users to choose more documents they initially did not favor. Moreover, we find that the training data have notable impacts on the manipulation degrees, and algorithms with more powerful modeling abilities are more sensitive to such impacts. The experiments also verified the usefulness of the proposed metrics for measuring the degree of manipulations. We advocate that future recommendation algorithm studies should be treated as an optimization problem with constrained user preference manipulations.

Assimilation and Contrast: the Two-sided Anchoring Effects of Recommender Systems

Effects of the Conversation and Recommendation Mechanism on Chatbots' Recommendation Effectiveness

Nonmonetary Rewards of Referral Reward Programs and Recommendation Intention: the Role of Reward–product Congruity

Complements Are Warm and Substitutes Are Competent: the Effect of Recommendation Type on Focal Product Evaluation.

To Switch or Not To Switch: Understanding Social Influence in Recommender Systems

Anchoring Bias in Online Voting

How Do Recommender Systems Lead to Consumer Purchases? A Causal Mediation Analysis of a Field Experiment

Comprehension and Assessment of Product Reviews: A Review-Product Congruity Proposition.

First Things First? Order Effects in Online Product Recommender Systems

Retention Induced Biases in a Recommendation System with Heterogeneous Users

The Effect of Incentive Structure on Referral: the Determining Role of Self-Construal

A Dual Process Model of the Influence of Recommender Systems on Purchase Intentions in Online Shopping Environments

Helpfulness of Online Review Content: the Moderating Effects of Temporal and Social Cues

Producer-Side Experiments Based on Counterfactual Interleaving Designs for Online Recommender Systems

Non-Compensatory Psychological Models for Recommender Systems.

Status reinforcing: Unintended rating bias on online shopping platforms

Evaluating Trust in Recommender Systems: A User Study on the Impacts of Explanations, Agency Attribution, and Product Types

Understanding the Substitution Effect Between Online and Traditional Channels: Evidence from Product Attributes Perspective.

On the Differences Between View-Based and Purchase-Based Recommender Systems

Understanding or Manipulation: Rethinking Online Performance Gains of Modern Recommender Systems

To Recommend or Not? A Model-Based Comparison of Item-Matching Processes