Abstract: External knowledge,e.g., entities and entity descriptions, can help humans understand texts. Many works have been explored to include external knowledge in the pre-trained models. These methods, generally, design pre-training tasks and implicitly introduce knowledge by updating model weights, alternatively, use it straightforwardly together with the original text. Though effective, there are some limitations. On the one hand, it is implicit and only model weights are paid attention to, the pre-trained entity embeddings are ignored. On the other hand, entity descriptions may be lengthy, and inputting into the model together with the original text may distract the model's attention. This paper aims to explicitly include both entities and entity descriptions in the fine-tuning stage. First, the pre-trained entity embeddings are fused with the original text representation and updated by the backbone model layer by layer. Second, descriptions are represented by the knowledge module outside the backbone model, and each knowledge layer is selectively connected to one backbone layer for fusing. Third, two knowledge-related auxiliary tasks, i.e., entity/description enhancement and entity enhancement/pollution task, are designed to smooth the semantic gaps among evolved representations. We conducted experiments on four knowledge-oriented tasks and two common tasks, and the results achieved new state-of-the-art on several datasets. Besides, we conduct an ablation study to show that each module in our method is necessary. The code is available at https://github.com/lshowway/Ered.

Entity-Duet Neural Ranking: Understanding the Role of Knowledge Graph Semantics in Neural Information Retrieval.

Word-Entity Duet Representations for Document Ranking

DREQ: Document Re-Ranking Using Entity-based Query Understanding

Graph-Embedding Empowered Entity Retrieval

Neural IR Meets Graph Embedding: A Ranking Model for Product Search

Explore Entity Embedding Effectiveness in Entity Retrieval

Representation Learning of Knowledge Graphs with Entity Descriptions

Explicit Semantic Ranking for Academic Search via Knowledge Graph Embedding

Entity-aware Collaborative Relation Network with Knowledge Graph for Recommendation

Distributed Representation Learning for Knowledge Graphs with Entity Descriptions

Learning Better Representations for Neural Information Retrieval with Graph Information

DeepE: a deep neural network for knowledge graph embedding

Understanding and Improving Knowledge Graph Embedding for Entity Alignment

EntityLDA: A Topic Model for Entity Retrieval on Knowledge Graph

Neural Recurrent Structure Search for Knowledge Graph Embedding

A Knowledge Graph Entity Disambiguation Method Based on Entity-Relationship Embedding and Graph Structure Embedding

Joint Representations of Text and Knowledge Graphs for Retrieval and Evaluation

EAGER: Embedding-Assisted Entity Resolution for Knowledge Graphs

NERank+: a Graph-Based Approach for Entity Ranking in Document Collections.

Ered: Enhanced Text Representations with Entities and Descriptions

Learning to Explain Entity Relationships by Pairwise Ranking with Convolutional Neural Networks.