Abstract:Recommender systems (RSs) are designed to provide personalized recommendations to users. Recently, knowledge graphs (KGs) have been widely introduced in RSs to improve recommendation accuracy. In this study, however, we demonstrate that RSs do not necessarily perform worse even if the KG is downgraded to the user-item interaction graph only (or removed). We propose an evaluation framework KG4RecEval to systematically evaluate how much a KG contributes to the recommendation accuracy of a KG-based RS, using our defined metric KGER (KG utilization efficiency in recommendation). We consider the scenarios where knowledge in a KG gets completely removed, randomly distorted and decreased, and also where recommendations are for cold-start users. Our extensive experiments on four commonly used datasets and a number of state-of-the-art KG-based RSs reveal that: to remove, randomly distort or decrease knowledge does not necessarily decrease recommendation accuracy, even for cold-start users. These findings inspire us to rethink how to better utilize knowledge from existing KGs, whereby we discuss and provide insights into what characteristics of datasets and KG-based RSs may help improve KG utilization efficiency.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: How much of a role does the knowledge graph (KG) actually play in KG - based recommender systems (RSs)? Specifically, the authors question whether the accuracy of the recommender system will be affected if the knowledge in the knowledge graph is removed, distorted, or reduced. In addition, they also explore whether the role of the knowledge graph is different for cold - start users (i.e., new users or new items). ### Summary of research questions: 1. **RQ1**: What if there is no knowledge graph? 2. **RQ2**: What if the knowledge is false? 3. **RQ3**: What if the knowledge is reduced? 4. **RQ4**: For cold - start users, what is the role of the knowledge graph? 5. **RQ5**: Which has a greater impact on recommendation accuracy, the authenticity or the quantity of knowledge? ### Main contributions: - **System evaluation framework**: Designed an evaluation framework named KG4R/e.sc/c.scE/v.sc/a.sc/l.sc for systematically evaluating the impact of the knowledge graph on KG - based recommender systems. - **Proposing the KGER index**: Defined a new evaluation index - KG Utilization Efficiency in Recommendation (KGER) to quantify the impact of the knowledge graph on recommendation accuracy. - **Experimental verification**: Conducted extensive experiments with multiple datasets and the latest KG - based recommender systems, revealing some counter - intuitive results. ### Key findings of experimental results: - **Removing or distorting the knowledge graph**: Even if the knowledge in the knowledge graph is removed or distorted, the accuracy of the recommender system does not necessarily decrease, even for cold - start users. - **The influence of different datasets and models**: Different datasets and recommender system models have different degrees of dependence on the knowledge graph, indicating that the role of KG is highly dependent on the specific application scenario. - **The influence of authenticity and quantity**: For ordinary users, the authenticity and quantity of knowledge have a similar impact on recommendation accuracy; for cold - start users, the quantity of knowledge is slightly more influential. ### Conclusion: By answering these research questions, the authors challenge the previous conclusion that the knowledge graph can significantly improve the performance of the recommender system and provide implications for future research. Overall, this paper prompts us to rethink how to use the knowledge in the existing knowledge graph more effectively to improve the performance of the recommender system. ### Formula representation: The formulas involved in the paper are as follows: - **KGER formula**: \[ \text{KGER}(S, G, M, \Delta)=\frac{M(S, G)-M(S,(G - \Delta))}{\Delta\cdot M(S, G)} \] where: - \(S\) represents the recommendation model, - \(G\) represents the knowledge graph, - \(M\) represents the index for evaluating recommendation performance (e.g., MRR), - \(\Delta\) represents the proportion of the reduction in real knowledge. Hopefully, this information will help you better understand the main content and findings of this paper. If you have more questions, feel free to continue asking!

Does Knowledge Graph Really Matter for Recommender Systems?

A Survey on Knowledge Graph-Based Recommender Systems

Unleashing the Power of Knowledge Graph for Recommendation Via Invariant Learning

Knowledge Graph Preference Migration Network for Recommendation.

DSKReG: Differentiable Sampling on Knowledge Graph for Recommendation with Relational GNN

EditKG: Editing Knowledge Graph for Recommendation

Deep Learning on Knowledge Graph for Recommender System: A Survey

Knowledge Graph Pruning for Recommendation

Attentive Knowledge-aware Graph Convolutional Networks with Collaborative Guidance for Personalized Recommendation

Knowledge Graph Random Neural Networks for Recommender Systems

Knowledge-Aware Hypergraph Neural Network for Recommender Systems

Research on intelligent recommendation algorithm of literature based on knowledge graph technology

How to Surprisingly Consider Recommendations? A Knowledge-Graph-based Approach Relying on Complex Network Metrics

Rethinking Knowledge Graph Evaluation Under the Open-World Assumption

Reinforced Negative Sampling over Knowledge Graph for Recommendation

Exploring High-Order User Preference with Knowledge Graph for Recommendation

Aggregating knowledge-aware graph neural network and adaptive relational attention for recommendation

Reinforcement Learning-Based Recommendation with User Reviews on Knowledge Graphs.

Graph-based Non-Sampling for Knowledge Graph Enhanced Recommendation

Community Enhanced Knowledge Graph for Recommendation

KGCFRec: Improving Collaborative Filtering Recommendation with Knowledge Graph