Abstract:Recommender systems are expected to be assistants that help human users find relevant information automatically without explicit queries. As recommender systems evolve, increasingly sophisticated learning techniques are applied and have achieved better performance in terms of user engagement metrics such as clicks and browsing time. The increase in the measured performance, however, can have two possible attributions: a better understanding of user preferences, and a more proactive ability to utilize human bounded rationality to seduce user over-consumption. A natural following question is whether current recommendation algorithms are manipulating user preferences. If so, can we measure the manipulation level? In this paper, we present a general framework for benchmarking the degree of manipulations of recommendation algorithms, in both slate recommendation and sequential recommendation scenarios. The framework consists of four stages, initial preference calculation, training data collection, algorithm training and interaction, and metrics calculation that involves two proposed metrics. We benchmark some representative recommendation algorithms in both synthetic and real-world datasets under the proposed framework. We have observed that a high online click-through rate does not necessarily mean a better understanding of user initial preference, but ends in prompting users to choose more documents they initially did not favor. Moreover, we find that the training data have notable impacts on the manipulation degrees, and algorithms with more powerful modeling abilities are more sensitive to such impacts. The experiments also verified the usefulness of the proposed metrics for measuring the degree of manipulations. We advocate that future recommendation algorithm studies should be treated as an optimization problem with constrained user preference manipulations.

Recommendation Algorithms for Optimizing Hit Rate, User Satisfaction and Website Revenue

A Hybrid Recommendation Algorithm Based on Clustering and Collaborative Filtering

User Interest Dynamics on Personalized Recommendation

An Interactive Personalized Recommendation System Using the Hybrid Algorithm Model

An Improved Network-Based Recommendation Model Via Inhibiting Algorithm Bias

Hybrid Recommendation Algorithm Based On Two Roles Of Social Tags

Website-oriented recommendation based on heat spreading and tag-aware collaborative filtering

A Personalized Recommendation Algorithm Via Biased Random Walk

Value-aware Recommendation based on Reinforced Profit Maximization in E-commerce Systems

Interest Before Liking: Two-step Recommendation Approaches

A pareto-efficient algorithm for multiple objective optimization in e-commerce recommendation

Optimal Recommendation in the Presense of Comparison Shopping

We Know What You Want: An Advertising Strategy Recommender System for Online Advertising

Personalized Recommendation Algorithm Based on Product Reviews

How good your recommender system is? A survey on evaluations in recommendation

Understanding or Manipulation: Rethinking Online Performance Gains of Modern Recommender Systems

A Recommendation Approach Based on Heterogeneous Network and Dynamic Knowledge Graph

Incorporating Consumer Browse Data: Extended Item-Based Top-K Recommendation Algorithms

Lower-Left Partial AUC: An Effective and Efficient Optimization Metric for Recommendation

Towards Fair and Rigorous Evaluations: Hyperparameter Optimization for Top-N Recommendation Task with Implicit Feedback

Show Me the Money: Dynamic Recommendations for Revenue Maximization