Abstract:In real-world scenarios, most of the data obtained from information retrieval (IR) system is unstructured. Converting natural language sentences into structured Knowledge Graphs (KGs) remains a critical challenge. The quality of constructed KGs may also impact the performance of some KG-dependent domains like GraphRAG systems and recommendation systems. Recently, Large Language Models (LLMs) have demonstrated impressive capabilities in addressing a wide range of natural language processing tasks. However, there are still challenges when utilizing LLMs to address the task of generating structured KGs. And we have identified three limitations with respect to existing KG construction methods. (1)There is a large amount of information and excessive noise in real-world documents, which could result in extracting messy information. (2)Native LLMs struggle to effectively extract accuracy knowledge from some domain-specific documents. (3)Hallucinations phenomenon cannot be overlooked when utilizing LLMs directly as an unsupervised method for constructing KGs. In this paper, we propose GraphJudger, a knowledge graph construction framework to address the aforementioned challenges. We introduce three innovative modules in our method, which are entity-centric iterative text denoising, knowledge aware instruction tuning and graph judgement, respectively. We seek to utilize the capacity of LLMs to function as a graph judger, a capability superior to their role only as a predictor for KG construction problems. Experiments conducted on two general text-graph pair datasets and one domain-specific text-graph pair dataset show superior performances compared to baseline methods. The code of our proposed method is available at <a class="link-external link-https" href="https://github.com/hhy-huang/GraphJudger" rel="external noopener nofollow">this https URL</a>.

Knowledge Graph Accuracy Evaluation: an LLM-enhanced Embedding Approach

A Survey on Knowledge Graph Embedding: Approaches, Applications and Benchmarks

Knowledge Graph-Enhanced Large Language Models via Path Selection

Fine-Grained Evaluation of Knowledge Graph Embedding Model in Knowledge Enhancement Downstream Tasks

Evaluating Knowledge Graph Accuracy Powered by Optimized Human-machine Collaboration

Can LLMs be Good Graph Judger for Knowledge Graph Construction?

Exploring Large Language Models for Knowledge Graph Completion

Construction of Legal Knowledge Graph Based on Knowledge-Enhanced Large Language Models

Efficient Knowledge Infusion via KG-LLM Alignment

Knowledge Graph Embedding Based on Hyperplane and Quantitative Credibility.

Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs

KGLens: Towards Efficient and Effective Knowledge Probing of Large Language Models with Knowledge Graphs

A knowledge graph embedding model based attention mechanism for enhanced node information integration

Learning Structured Embeddings of Knowledge Graphs with Generative Adversarial Framework

Knowledge Graph Embedding: A Survey of Approaches and Applications

Efficient knowledge graph accuracy evaluation

Towards Robust Knowledge Graph Embedding via Multi-task Reinforcement Learning

Empowering Small-Scale Knowledge Graphs: A Strategy of Leveraging General-Purpose Knowledge Graphs for Enriched Embeddings

Knowledge Enhanced Fact Checking and Verification

Knowledge Graph Embedding Via Graph Attenuated Attention Networks