Abstract:Recognizing relations between two words is a fundamental task with the broad applications. Different from extracting relations from text, it is difficult to identify relations among words without their contexts. Especially for long-tail relations, it becomes more difficult due to inadequate semantic features. Existing approaches based on language models (LMs) utilize rich knowledge of LMs to enhance the semantic features of relations. However, they capture uncommon relations while overlooking less frequent but meaningful ones since knowledge of LMs seriously relies on trained data where often represents common relations. On the other hand, long-tail relations are often uncommon in training data. It is interesting but not trivial to use external knowledge to enrich LMs due to collecting corpus containing long-tail relationships is hardly feasible. In this paper, we propose a sememe knowledge enhanced method (SememeLM) to enhance the representation of long-tail relations, in which sememes can break the contextual constraints between wors. Firstly, we present a sememe relation graph and propose a graph encoding method. Moreover, since external knowledge base possibly consisting of massive irrelevant knowledge, the noise is introduced. We propose a consistency alignment module, which aligns the introduced knowledge with LMs, reduces the noise and integrates the knowledge into the language model. Finally, we conducted experiments on word analogy datasets, which evaluates the ability to distinguish relation representations subtle differences, including long-tail relations. Extensive experiments show that our approach outperforms some state-of-the-art methods.

Sememe Knowledge Representation

Sememe Knowledge Computation: a Review of Recent Advances in Application and Expansion of Sememe Knowledge Bases

OpenHowNet: An Open Sememe-based Lexical Knowledge Base.

Modeling Semantic Compositionality With Sememe Knowledge

Sememe Prediction: Learning Semantic Knowledge from Unstructured Textual Wiki Descriptions.

Chinese Lexical Sememe Prediction Using CilinE Knowledge

Improved Word Representation Learning with Sememes

Incorporating Chinese Characters of Words for Lexical Sememe Prediction

Automatic Construction of Sememe Knowledge Bases via Dictionaries.

SememeLM: A Sememe Knowledge Enhanced Method for Long-tail Relation Representation

SeMi: A SEmantic Modeling machIne to build Knowledge Graphs with graph neural networks

Predicting Categorial Sememe for English-Chinese Word Pairs via Representations in Explainable Sememe Space.

Sememe Tree Prediction for English-Chinese Word Pairs.

Towards Building a Multilingual Sememe Knowledge Base: Predicting Sememes for BabelNet Synsets

Lexical Sememe Prediction Via Word Embeddings and Matrix Factorization.

Try to Substitute: an Unsupervised Chinese Word Sense Disambiguation Method Based on HowNet.

Knowledge Graph Representation of Syntactic and Semantic Information.

Building Semantic Dependency Knowledge Graph Based on HowNet.

Learning the Implicit Semantic Representation on Graph-Structured Data

Sememe Prediction for BabelNet Synsets Using Multilingual and Multimodal Information

Enhancing Transformer with Sememe Knowledge.