Abstract:Textual representations play an important role in the field of natural language processing (NLP). The efficiency of NLP tasks, such as text comprehension and information extraction, can be significantly improved with proper textual representations. As neural networks are gradually applied to learn the representation of words and phrases, fairly efficient models of learning short text representations have been developed, such as the continuous bag of words (CBOW) and skip-gram models, and they have been extensively employed in a variety of NLP tasks. Because of the complex structure generated by the longer text lengths, such as sentences, algorithms appropriate for learning short textual representations are not applicable for learning long textual representations. One method of learning long textual representations is the Long Short-Term Memory (LSTM) network, which is suitable for processing sequences. However, the standard LSTM does not adequately address the primary sentence structure (subject, predicate and object), which is an important factor for producing appropriate sentence representations. To resolve this issue, this paper proposes the dependency-based LSTM model (D-LSTM). The D-LSTM divides a sentence representation into two parts: a basic component and a supporting component. The D-LSTM uses a pre-trained dependency parser to obtain the primary sentence information and generate supporting components, and it also uses a standard LSTM model to generate the basic sentence components. A weight factor that can adjust the ratio of the basic and supporting components in a sentence is introduced to generate the sentence representation. Compared with the representation learned by the standard LSTM, the sentence representation learned by the D-LSTM contains a greater amount of useful information. The experimental results show that the D-LSTM is superior to the standard LSTM for sentences involving compositional knowledge (SICK) data.

DWLTI Method Based on Deepwalk with Limited Text Information

Dependency-based Siamese Long Short-Term Memory Network for Learning Sentence Representations.

Network Representation Learning with Rich Text Information

Study on text representation method based on deep learning and topic information

Discriminative Deep Random Walk for Network Classification.

The Network Representation Learning Algorithm Based on Semi-Supervised Random Walk

Hybrid Embedding via Cross-Layer Random Walks on Multiplex Networks

WalkLM: A Uniform Language Model Fine-tuning Framework for Attributed Graph Embedding

BTWalk: Branching Tree Random Walk for Multi-Order Structured Network Embedding

Network Embedding As Matrix Factorization: Unifying DeepWalk, LINE, PTE, and Node2vec

Max-margin Deepwalk: Discriminative Learning of Network Representation

Semantic-aware network embedding via optimized random walk and paragaraph2vec

A modified DeepWalk method for link prediction in attributed social network

Language Clustering with Word Co-Occurrence Networks Based on Parallel Texts

Sampled in Pairs and Driven by Text: A New Graph Embedding Framework

DeepWalk: Online Learning of Social Representations

Comprehend DeepWalk as Matrix Factorization

Link prediction in multilayer networks using weighted reliable local random walk algorithm

Lightweight Text Matching Method with Rich Features.

Link prediction in heterogeneous information networks: An improved deep graph convolution approach

Network Embedding for Graphs with Node Attributes