Knowledge Graph Completing with Dual Confrontation Learning Model Based on Variational Information Bottleneck Method
Song Han,Zhengyi Guan,Sihui Li,Jin Wang,Xiaobing Zhou
DOI: https://doi.org/10.1109/qrs60937.2023.00077
2023-01-01
Abstract:In natural language learning, pre-trained language models (PLM) can acquire rich knowledge and concepts from rich corpora, making it possible to use PLM-based models for knowledge graph completion (KGC) tasks. However, in previous research, when applying pre-trained models to knowledge graph completion tasks, two main challenges persist: (1) Existing knowledge graph completion models are typically evaluated based on the closed-world assumption(CWA), thus lacking evaluation methods suitable for the open-world assumption(OWA), which constitutes a significant challenge in the current field of knowledge graph completion. (2) Extracting useful information, reducing noise, and providing clear interpretability for extracting effective information from the extensive prior knowledge embedded in pre-trained language models is also a crucial issue. Although the loss function can reduce noise to a certain extent, from the perspective of information theory, only relying on the loss function has a limited effect on noise reduction, and the model needs more professional tools to reduce noise and reduce the impact of irrelevant information on model performance. To address the aforementioned challenges, we propose a dual confrontation learning model based on the variational information bottleneck method. This model restricts information flow and feature selection from the perspective of information theory to reduce noise and enhance model performance while providing clear interpretability for this process. Based on extensive experiments and comprehensive evaluations conducted under both closed-world and open-world assumptions, this model successfully extracts valuable knowledge from pre-trained language models to accomplish KGC tasks. Simultaneously, it minimizes noise, removes non-robust features, enhances model reliability, and optimizes model performance. More importantly, we offer a strong interpretability for the process in which our model constrains information flow to reduce noise.