Knowledge Graphs as Context Sources for LLM-Based Explanations of Learning Recommendations

Hasan Abu-Rasheed,Christian Weber,Madjid Fathi

2024-03-05

Abstract:In the era of personalized education, the provision of comprehensible explanations for learning recommendations is of a great value to enhance the learner's understanding and engagement with the recommended learning content. Large language models (LLMs) and generative AI in general have recently opened new doors for generating human-like explanations, for and along learning recommendations. However, their precision is still far away from acceptable in a sensitive field like education. To harness the abilities of LLMs, while still ensuring a high level of precision towards the intent of the learners, this paper proposes an approach to utilize knowledge graphs (KG) as a source of factual context, for LLM prompts, reducing the risk of model hallucinations, and safeguarding against wrong or imprecise information, while maintaining an application-intended learning context. We utilize the semantic relations in the knowledge graph to offer curated knowledge about learning recommendations. With domain-experts in the loop, we design the explanation as a textual template, which is filled and completed by the LLM. Domain experts were integrated in the prompt engineering phase as part of a study, to ensure that explanations include information that is relevant to the learner. We evaluate our approach quantitatively using Rouge-N and Rouge-L measures, as well as qualitatively with experts and learners. Our results show an enhanced recall and precision of the generated explanations compared to those generated solely by the GPT model, with a greatly reduced risk of generating imprecise information in the final learning explanation.

Artificial Intelligence

What problem does this paper attempt to address?

This paper aims to address the problem of providing accurate and understandable explanations for learning recommendation automatically. Current Large Language Models (LLMs) lack precision in generating such explanations and may produce erroneous information. The paper proposes using a Knowledge Graph (KG) as a factual context source to reduce LLMs' erroneous generation (hallucinations), ensuring the accuracy of explanations while maintaining educational relevance. By involving experts, text templates are designed and filled in with LLMs to improve the recall and precision of generating explanations. Experimental results show that combining KG with the GPT model can generate more accurate explanations and reduce the risk of irrelevant or inaccurate information compared to using the GPT model alone.

Knowledge Graphs as Context Sources for LLM-Based Explanations of Learning Recommendations

LLM-Powered Explanations: Unraveling Recommendations Through Subgraph Reasoning

Faithfully Explainable Recommendation via Neural Logic Reasoning

Knowledge-grounded Natural Language Recommendation Explanation

Building Contextual Knowledge Graphs for Personalized Learning Recommendations using Text Mining and Semantic Graph Completion

Knowledge Enhanced Graph Neural Networks for Explainable Recommendation

Supporting Student Decisions on Learning Recommendations: An LLM-Based Chatbot with Knowledge Graph Contextualization for Conversational Explainability and Mentoring

Explainable Entity-based Recommendations with Knowledge Graphs

LMExplainer: Grounding Knowledge and Explaining Language Models

Contextualized Knowledge Graph Embedding for Explainable Talent Training Course Recommendation

Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphs

Knowledge-aware Reasoning with Self-Supervised Reinforcement Learning for Explainable Recommendation in MOOCs

LLM is Knowledge Graph Reasoner: LLM's Intuition-aware Knowledge Graph Reasoning for Cold-start Sequential Recommendation

XplainLLM: A Knowledge-Augmented Dataset for Reliable Grounded Explanations in LLMs

Enabling Explainable Recommendation in E-commerce with LLM-powered Product Knowledge Graph

Multi-modal Knowledge-aware Reinforcement Learning Network for Explainable Recommendation

Context-Aware Explainable Recommendation Based on Domain Knowledge Graph

LLM-Guided Multi-View Hypergraph Learning for Human-Centric Explainable Recommendation

Finding Paths for Explainable MOOC Recommendation: A Learner Perspective

LLMRG: Improving Recommendations through Large Language Model Reasoning Graphs

KR-GCN: Knowledge-aware Reasoning with Graph Convolution Network for Explainable Recommendation