Abstract:Human-curated knowledge graphs provide critical supportive information to various natural language processing tasks, but these graphs are usually incomplete, urging auto-completion of them (a.k.a. knowledge graph completion). Prevalent graph embedding approaches, e.g., TransE, learn structured knowledge via representing graph elements (i.e., entities/relations) into dense embeddings and capturing their triple-level relationship with spatial distance. However, they are hardly generalizable to the elements never visited in training and are intrinsically vulnerable to graph incompleteness. In contrast, textual encoding approaches, e.g., KG-BERT, resort to graph triple’s text and triple-level contextualized representations. They are generalizable enough and robust to the incompleteness, especially when coupled with pre-trained encoders. But two major drawbacks limit the performance: (1) high overheads due to the costly scoring of all possible triples in inference, and (2) a lack of structured knowledge in the textual encoder. In this paper, we follow the textual encoding paradigm and aim to alleviate its drawbacks by augmenting it with graph embedding techniques – a complementary hybrid of both paradigms. Specifically, we partition each triple into two asymmetric parts as in translation-based graph embedding approach, and encode both parts into contextualized representations by a Siamese-style textual encoder. Built upon the representations, our model employs both deterministic classifier and spatial measurement for representation and structure learning respectively. It thus reduces the overheads by reusing graph elements’ embeddings to avoid combinatorial explosion, and enhances structured knowledge by exploring the spatial characteristics. Moreover, we develop a self-adaptive ensemble scheme to further improve the performance by incorporating triple scores from an existing graph embedding model. In experiments, we achieve state-of-the-art performance on three benchmarks and a zero-shot dataset for link prediction, with highlights of inference costs reduced by 1-2 orders of magnitude compared to a sophisticated textual encoding method.

Text-Augmented Knowledge Representation Learning Based on Convolutional Network

Knowledge Representation Learning with Entity Descriptions, Hierarchical Types, and Textual Relations

Incorporating Knowledge into Neural Network for Text Representation.

Text-Enhanced Knowledge Representation Learning Based on Gated Convolutional Networks

Convolutional Network Embedding of Text-Enhanced Representation for Knowledge Graph Completion

A Knowledge Graph Embedding Method Based on Neural Network

Learning Knowledge Graph Embedding with a Bi-Directional Relation Encoding Network and a Convolutional Autoencoder Decoding Network

Knowledge Representation Via Joint Learning of Sequential Text and Knowledge Graphs.

Knowledge Graph Embedding Using a Multi-Channel Interactive Convolutional Neural Network with Triple Attention

A Model of Text-Enhanced Knowledge Graph Representation Learning with Mutual Attention.

Convolutional Models with Multi-Feature Fusion for Effective Link Prediction in Knowledge Graph Embedding

Text-Enhanced and Relational Context Based Hyperbolic Knowledge Graph Embedding

Knowledge Graph Representation with Jointly Structural and Textual Encoding

HyConvE: A Novel Embedding Model for Knowledge Hypergraph Link Prediction with Convolutional Neural Networks

Fusing Structural Information with Knowledge Enhanced Text Representation for Knowledge Graph Completion

Knowledge graph embedding model with attention-based high-low level features interaction convolutional network

Knowledge Graph Completion Based on the Joint Embedding of Entities

Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion

ResConvE: Deeper Convolution-Based Knowledge Graph Embeddings

Text-Graph Enhanced Knowledge Graph Representation Learning.

A Representation Learning Method of Knowledge Graph Integrating Relationship Path, Text Description and Entity Hierarchy Type Information