Abstract:Knowledge Graphs (KGs) provide supportively structured knowledge and have been applied to various downstream applications. Given a large amount of incomplete knowledge in KGs, knowledge graph completion (KGC) is proposed to reason over known facts and infer the missing links. The previous graph embedding approaches learn graph structure (i.e., triple structure/neighborhood structure) but cannot handle unseen entities, which is addressed by textual encoding approaches that utilize the textual knowledge of graph elements (i.e., entities/relations). However, the previous textual encoding approaches only resort to triples and thus cannot exploit the knowledge of neighbors, which provides abundant evidence to facilitate prediction. Moreover, they are insensitive to changes in the position of elements in triples when performing text modeling, and thus cannot effectively distinguish triples with the same elements but completely different semantics, which is detrimental to the final result. To address the above challenges, we propose a novel Structure-Enhanced and Position-Aware Knowledge Embedding (SEPAKE) framework. Specifically, masked elements reconstruction is devised to predict missing elements by reasoning over the contexts of subgraphs. As such, we incorporate the graph structure while maintaining the feature that textual information can be encoded. Meanwhile, position-aware learning is conducted to capture the semantic knowledge implied by the relative positions of elements in textualization. In addition, we employ task-specific adapters to store knowledge in a unified way to facilitate the storage and transfer of knowledge. Extensive experiments demonstrate the effectiveness of our framework, and we achieve state-of-the-art performance on standard datasets compared with textual encoding approaches. Besides, our proposed framework can efficiently improve the previous approaches by optionally pluggable adapters, further verifying the advancement and applicability of our work.

Bridge Text and Knowledge by Learning Multi-Prototype Entity Mention Embedding

Boosting Collective Entity Linking via Type-Guided Semantic Embedding.

Knowledge Representation Learning with Entity Descriptions, Hierarchical Types, and Textual Relations

Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extraction.

Do Multi-Sense Embeddings Improve Natural Language Understanding?

Bridge: A Unified Framework to Knowledge Graph Completion via Language Models and Knowledge Representation

Modeling Mention, Context and Entity with Neural Networks for Entity Disambiguation.

Attention-Based Joint Entity Linking with Entity Embedding

On Modeling Sense Relatedness in Multi-prototype Word Embedding.

Exploiting Global Semantic Similarities in Knowledge Graphs by Relational Prototype Entities

Few-shot named entity recognition with hybrid multi-prototype learning

SPOT: Knowledge-Enhanced Language Representations for Information Extraction

Integrating K+ Entities into Coreference Resolution on Biomedical Texts

Joint Representation Learning of Text and Knowledge for Knowledge Graph Completion

Contextualise Entities and Relations: An Interaction Method for Knowledge Graph Completion

SEPAKE: a structure-enhanced and position-aware knowledge embedding framework for knowledge graph completion

Bridge inspection named entity recognition via BERT and lexicon augmented machine reading comprehension neural model

Multi-information Embedding Based Entity Alignment.

Knowledge-Enhanced Bilingual Textual Representations for Cross-Lingual Semantic Textual Similarity

Entity Disambiguation via Fusion Entity Decoding

Multi-phase Word Sense Embedding Retrofitting with Lexical Ontology