Abstract:Recommender systems are expected to be assistants that help human users find relevant information automatically without explicit queries. As recommender systems evolve, increasingly sophisticated learning techniques are applied and have achieved better performance in terms of user engagement metrics such as clicks and browsing time. The increase in the measured performance, however, can have two possible attributions: a better understanding of user preferences, and a more proactive ability to utilize human bounded rationality to seduce user over-consumption. A natural following question is whether current recommendation algorithms are manipulating user preferences. If so, can we measure the manipulation level? In this paper, we present a general framework for benchmarking the degree of manipulations of recommendation algorithms, in both slate recommendation and sequential recommendation scenarios. The framework consists of four stages, initial preference calculation, training data collection, algorithm training and interaction, and metrics calculation that involves two proposed metrics. We benchmark some representative recommendation algorithms in both synthetic and real-world datasets under the proposed framework. We have observed that a high online click-through rate does not necessarily mean a better understanding of user initial preference, but ends in prompting users to choose more documents they initially did not favor. Moreover, we find that the training data have notable impacts on the manipulation degrees, and algorithms with more powerful modeling abilities are more sensitive to such impacts. The experiments also verified the usefulness of the proposed metrics for measuring the degree of manipulations. We advocate that future recommendation algorithm studies should be treated as an optimization problem with constrained user preference manipulations.

Manipulation Robustness of Collaborative Filtering Systems

Using latent class models for neighbors selection in collaborative filtering

Improving Recommendations With Collaborative Factors

A unified latent factor correction scheme for collaborative filtering

Brand Recommendation Leveraging Heterogeneous Implicit Feedbacks

Data Poisoning Attacks on Factorization-Based Collaborative Filtering

Robust enhanced collaborative filtering without explicit noise filtering

Enhancing the Robustness of Neural Collaborative Filtering Systems under Malicious Attacks

Detection of Shilling Attacks in Collaborative Filtering Recommender Systems.

Deconfounded Causal Collaborative Filtering

Online Collaborative Filtering With Implicit Feedback

Understanding or Manipulation: Rethinking Online Performance Gains of Modern Recommender Systems

Analysis of Shilling Attacks on SVD-based Collaborative Filtering Algorithm

A Comparative Study of Collaborative Filtering Algorithms

Stability of matrix factorization for collaborative filtering

An Adaptive Collaborative Filtering Algorithm for Online Reputation Systems.

Scalable collaborative filtering using cluster-based smoothing.

Fraud Detection in Online Consumer Reviews

A Survey of Collaborative Filtering Techniques

Kernel-CF: Collaborative filtering done right with social network analysis and kernel smoothing

Causal Collaborative Filtering