Abstract:Collaborative filtering models based on matrix factorization and learned similarities using Artificial Neural Networks (ANNs) have gained significant attention in recent years. This is, in part, because ANNs have demonstrated good results in a wide variety of recommendation tasks. The introduction of ANNs within the recommendation ecosystem has been recently questioned, raising several comparisons in terms of efficiency and effectiveness. One aspect most of these comparisons have in common is their focus on accuracy, neglecting other evaluation dimensions important for the recommendation, such as novelty, diversity, or accounting for biases. We replicate experiments from three papers that compare Neural Collaborative Filtering (NCF) and Matrix Factorization (MF), to extend the analysis to other evaluation dimensions. Our contribution shows that the experiments are entirely reproducible, and we extend the study including other accuracy metrics and two statistical hypothesis tests. We investigated the Diversity and Novelty of the recommendations, showing that MF provides a better accuracy also on the long tail, although NCF provides a better item coverage and more diversified recommendations. We discuss the bias effect generated by the tested methods. They show a relatively small bias, but other recommendation baselines, with competitive accuracy performance, consistently show to be less affected by this issue. This is the first work, to the best of our knowledge, where several evaluation dimensions have been explored for an array of SOTA algorithms covering recent adaptations of ANNs and MF. Hence, we show the potential these techniques may have on beyond-accuracy evaluation while analyzing the effect on reproducibility these complementary dimensions may spark. Available at <a class="link-external link-http" href="http://github.com/sisinflab/Reenvisioning-the-comparison-between-Neural-Collaborative-Filtering-and-Matrix-Factorization" rel="external noopener nofollow">this http URL</a>

Collaborative Filtering vs. Content-Based Filtering: differences and similarities

A User-based and Item-based Collaborative Filtering Recommendation Algorithm

A Comparative Study of Collaborative Filtering Algorithms

Subjective Collaborative Filtering

A Comparative Study of Collaborative Filtering in Product Recommendation

Research on Collaborative Filtering Recommendation Algorithm

CF4CF: Recommending Collaborative Filtering algorithms using Collaborative Filtering

Comparative analysis of collaborative filtering techniques for the multi-criteria recommender systems

A Survey of Collaborative Filtering Techniques

A Comprehensive Review on Non-Neural Networks Collaborative Filtering Recommendation Systems

An Improved Collaborative Filtering Recommendation Algorithm and Recommendation Strategy

Comparative analysis of collaborative filtering recommender system algorithms for e-commerce

User Trust in Recommendation Systems: A comparison of Content-Based, Collaborative and Demographic Filtering

An improved content based collaborative filtering algorithm for movie recommendations

Using content features to enhance performance of user-based collaborative filtering performance of user-based collaborative filtering

Collaborative Filtering Algorithm Evaluation for Various Datasets

Reenvisioning Collaborative Filtering vs Matrix Factorization

Personalized Recommendations of Products to Users

Performance of Recommender Systems: Based on Content Navigator and Collaborative Filtering

Users' Rating Predictions Using Collaborating Filtering Based on Users and Items Similarity Measures