Abstract:In real-world scenarios, most of the data obtained from information retrieval (IR) system is unstructured. Converting natural language sentences into structured Knowledge Graphs (KGs) remains a critical challenge. The quality of constructed KGs may also impact the performance of some KG-dependent domains like GraphRAG systems and recommendation systems. Recently, Large Language Models (LLMs) have demonstrated impressive capabilities in addressing a wide range of natural language processing tasks. However, there are still challenges when utilizing LLMs to address the task of generating structured KGs. And we have identified three limitations with respect to existing KG construction methods. (1)There is a large amount of information and excessive noise in real-world documents, which could result in extracting messy information. (2)Native LLMs struggle to effectively extract accuracy knowledge from some domain-specific documents. (3)Hallucinations phenomenon cannot be overlooked when utilizing LLMs directly as an unsupervised method for constructing KGs. In this paper, we propose GraphJudger, a knowledge graph construction framework to address the aforementioned challenges. We introduce three innovative modules in our method, which are entity-centric iterative text denoising, knowledge aware instruction tuning and graph judgement, respectively. We seek to utilize the capacity of LLMs to function as a graph judger, a capability superior to their role only as a predictor for KG construction problems. Experiments conducted on two general text-graph pair datasets and one domain-specific text-graph pair dataset show superior performances compared to baseline methods. The code of our proposed method is available at <a class="link-external link-https" href="https://github.com/hhy-huang/GraphJudger" rel="external noopener nofollow">this https URL</a>.

Cross-Data Knowledge Graph Construction for LLM-enabled Educational Question-Answering System: A~Case~Study~at~HCMUT

Enhancing Large Language Models with Knowledge Graphs for Robust Question Answering

LLM4EduKG: LLM for Automatic Construction of Educational Knowledge Graph

LLM-Based Multi-Hop Question Answering with Knowledge Graph Integration in Evolving Environments

Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphs

Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering

LinkQ: An LLM-Assisted Visual Interface for Knowledge Graph Question-Answering

Multi-hop Question Answering over Knowledge Graphs using Large Language Models

Research on Intelligent Question-Answering Systems Based on Large Language Models and Knowledge Graphs

LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities

Document Knowledge Graph to Enhance Question Answering with Retrieval Augmented Generation

Enhancing Large Language Models with Pseudo- and Multisource- Knowledge Graphs for Open-ended Question Answering

Can LLMs be Good Graph Judger for Knowledge Graph Construction?

Retrieval and Reasoning on KGs: Integrate Knowledge Graphs into Large Language Models for Complex Question Answering

From human experts to machines: An LLM supported approach to ontology and knowledge graph construction

Reasoning on Efficient Knowledge Paths:Knowledge Graph Guides Large Language Model for Domain Question Answering

Efficient Knowledge Infusion via KG-LLM Alignment

HOLMES: Hyper-Relational Knowledge Graphs for Multi-hop Question Answering using LLMs

Graphusion: Leveraging Large Language Models for Scientific Knowledge Graph Fusion and Construction in NLP Education

An Intelligent Question Answering System of the Liao Dynasty Based on Knowledge Graph

Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs