Abstract:Recommender systems algorithms are generally evaluated primarily on machine learning criteria such as recommendation accuracy or top-n precision. In this work, we evaluate six recommendation algorithms from a user-centric perspective, collecting both objective user activity data and subjective user perceptions. In a field experiment involving 1508 users who participated for at least a month, we compare six algorithms built using machine learning techniques, ranging from supervised matrix factorization, contextual bandit learning to Q learning. We found that the objective design in machine-learning-based recommender systems significantly affects user experience. Specifically, a recommender optimizing for implicit action prediction error engages users more than optimizing for explicit rating prediction error when modeled with the classical matrix factorization algorithms, which empirically explains the historical transition of recommender system research from modeling explicit feedback data to implicit feedback data. However, the action-based recommender is not as precise as the rating-based recommender in that it increases not only positive engagement but also negative engagement, e.g., negative action rate and user browsing effort which are negatively correlated with user satisfaction. We show that blending both explicit and implicit feedback from users through an online learning algorithm can gain the benefits of engagement and mitigate one of the possible costs (i.e., the increased browsing effort).

Database Selection Based on Adaptively Implicit Feedback User Action Model

Intelligent Database Selection Based on the User Preferences

An Iterative Implicit Feedback Approach To Personalized Search

Personalization of Search Results Using Interaction Behaviors in Search Sessions.

Context-Based Adaptive Personalized Web Search for Improving Information Retrieval Effectiveness

Implicit Acquisition of Context for Personalization of Information Retrieval Systems

Still Haven't Found What You're Looking For -- Detecting the Intent of Web Search Missions from User Interaction Features

The Personalized Information Retrieval on the Basis of User-mode

Personalized Search: an Interactive and Iterative Approach.

Automatic Search Engine Evaluation Based On User Behavior Analysis

Personalized web search for improving retrieval effectiveness

Research on Personalized Search User Interest Updating Learning and Evaluation

Dynamic Analysis of Chinese Search Engine User Behavior

A Feedback-Based Web Search Engine

Automatic Search Engine Performance Evaluation Based on User Behavior Analysis

Using Online Relevance Feedback to Build Effective Personalized Metasearch Engine

Adaptive Multiple Feedback Strategies for Interactive Video Search

Development of a self-adaptive Web search engine

Explicit or implicit feedback? engagement or satisfaction?: a field experiment on machine-learning-based recommender systems.

Personalizing Search Engine Design on User's Interest

MetaSelector: Meta-Learning for Recommendation with User-Level Adaptive Model Selection