Abstract:Textual representations play an important role in the field of natural language processing (NLP). The efficiency of NLP tasks, such as text comprehension and information extraction, can be significantly improved with proper textual representations. As neural networks are gradually applied to learn the representation of words and phrases, fairly efficient models of learning short text representations have been developed, such as the continuous bag of words (CBOW) and skip-gram models, and they have been extensively employed in a variety of NLP tasks. Because of the complex structure generated by the longer text lengths, such as sentences, algorithms appropriate for learning short textual representations are not applicable for learning long textual representations. One method of learning long textual representations is the Long Short-Term Memory (LSTM) network, which is suitable for processing sequences. However, the standard LSTM does not adequately address the primary sentence structure (subject, predicate and object), which is an important factor for producing appropriate sentence representations. To resolve this issue, this paper proposes the dependency-based LSTM model (D-LSTM). The D-LSTM divides a sentence representation into two parts: a basic component and a supporting component. The D-LSTM uses a pre-trained dependency parser to obtain the primary sentence information and generate supporting components, and it also uses a standard LSTM model to generate the basic sentence components. A weight factor that can adjust the ratio of the basic and supporting components in a sentence is introduced to generate the sentence representation. Compared with the representation learned by the standard LSTM, the sentence representation learned by the D-LSTM contains a greater amount of useful information. The experimental results show that the D-LSTM is superior to the standard LSTM for sentences involving compositional knowledge (SICK) data.

Infusing Dependency Syntax Information into a Transformer Model for Document-Level Relation Extraction from Biomedical Literature

Document-Level Relation Extraction with a Dependency Syntax Transformer and Supervised Contrastive Learning

Dependency-based Siamese Long Short-Term Memory Network for Learning Sentence Representations.

Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models

A Syntax-enhanced Model Based on Category Keywords for Biomedical Relation Extraction

Discovering Medical Entity Relations from Texts using Dependency Information

A Long-Tail Relation Extraction Model Based on Dependency Path and Relation Graph Embedding

FAT-RE: A faster dependency-free model for relation extraction

Transformer-xl: Language modeling with longer-term dependency

Improve Discourse Dependency Parsing with Contextualized Representations

Dependency Position Encoding for Relation Extraction.

Document-Level Chemical-Induced Disease Semantic Relation Extraction Using Bidirectional Long Short-Term Memory on Dependency Graph

Syntactic Dependency for relation extraction from Biomedical literature

Exploring Effective Inter-Encoder Semantic Interaction for Document-Level Relation Extraction

Document-Level Relation Extraction with Structure Enhanced Transformer Encoder

Modeling Graph Structure in Transformer for Better AMR-to-Text Generation.

Relation Extraction in Biomedical Texts: A Cross-Sentence Approach

Hi-Transformer: Hierarchical Interactive Transformer for Efficient and Effective Long Document Modeling

Document-level Relation Extraction with Entity Interaction and Commonsense Knowledge

FCDS: Fusing Constituency and Dependency Syntax into Document-Level Relation Extraction

Leveraging Dependency Forest for Neural Medical Relation Extraction