Abstract:Textual representations play an important role in the field of natural language processing (NLP). The efficiency of NLP tasks, such as text comprehension and information extraction, can be significantly improved with proper textual representations. As neural networks are gradually applied to learn the representation of words and phrases, fairly efficient models of learning short text representations have been developed, such as the continuous bag of words (CBOW) and skip-gram models, and they have been extensively employed in a variety of NLP tasks. Because of the complex structure generated by the longer text lengths, such as sentences, algorithms appropriate for learning short textual representations are not applicable for learning long textual representations. One method of learning long textual representations is the Long Short-Term Memory (LSTM) network, which is suitable for processing sequences. However, the standard LSTM does not adequately address the primary sentence structure (subject, predicate and object), which is an important factor for producing appropriate sentence representations. To resolve this issue, this paper proposes the dependency-based LSTM model (D-LSTM). The D-LSTM divides a sentence representation into two parts: a basic component and a supporting component. The D-LSTM uses a pre-trained dependency parser to obtain the primary sentence information and generate supporting components, and it also uses a standard LSTM model to generate the basic sentence components. A weight factor that can adjust the ratio of the basic and supporting components in a sentence is introduced to generate the sentence representation. Compared with the representation learned by the standard LSTM, the sentence representation learned by the D-LSTM contains a greater amount of useful information. The experimental results show that the D-LSTM is superior to the standard LSTM for sentences involving compositional knowledge (SICK) data.

Improved Dependency Parsing Using Implicit Word Connections Learned from Unlabeled Data.

Dependency-based Siamese Long Short-Term Memory Network for Learning Sentence Representations.

Graph-based Dependency Parsing with Bidirectional LSTM.

Universal Dependencies Parsing For Colloquial Singaporean English

Integrating graph embedding and neural models for improving transition-based dependency parsing

Chinese Dependency Parsing Based on Treebank

Cross-lingual Dependency Parsing Based on Distributed Representations.

Enhanced Universal Dependency Parsing with Automated Concatenation of Embeddings

CSSL: Contrastive Self-Supervised Learning for Dependency Parsing on Relatively Free Word Ordered and Morphologically Rich Low Resource Languages

Enhancing Semantic Representations of Bilingual Word Embeddings with Syntactic Dependencies

A Distributed Representation-Based Framework for Cross-Lingual Transfer Parsing.

An Effective Neural Network Model for Graph-based Dependency Parsing.

Attention-based Belief or Disbelief Feature Extraction for Dependency Parsing.

Improving Chinese Dependency Parsing with Lexical Semantic Features

Leverage External Knowledge and Self-attention for Chinese Semantic Dependency Graph Parsing

Dependency-based Convolutional Neural Networks for Sentence Embedding

Incremental Parsing with Minimal Features Using Bi-Directional LSTM

Advances in Graph-based Dependency Parsing Using Deep Learning

Graph-Based Dependency Parsing With Recursive Neural Network

A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing

High-order Joint Constituency and Dependency Parsing