Abstract:Textual representations play an important role in the field of natural language processing (NLP). The efficiency of NLP tasks, such as text comprehension and information extraction, can be significantly improved with proper textual representations. As neural networks are gradually applied to learn the representation of words and phrases, fairly efficient models of learning short text representations have been developed, such as the continuous bag of words (CBOW) and skip-gram models, and they have been extensively employed in a variety of NLP tasks. Because of the complex structure generated by the longer text lengths, such as sentences, algorithms appropriate for learning short textual representations are not applicable for learning long textual representations. One method of learning long textual representations is the Long Short-Term Memory (LSTM) network, which is suitable for processing sequences. However, the standard LSTM does not adequately address the primary sentence structure (subject, predicate and object), which is an important factor for producing appropriate sentence representations. To resolve this issue, this paper proposes the dependency-based LSTM model (D-LSTM). The D-LSTM divides a sentence representation into two parts: a basic component and a supporting component. The D-LSTM uses a pre-trained dependency parser to obtain the primary sentence information and generate supporting components, and it also uses a standard LSTM model to generate the basic sentence components. A weight factor that can adjust the ratio of the basic and supporting components in a sentence is introduced to generate the sentence representation. Compared with the representation learned by the standard LSTM, the sentence representation learned by the D-LSTM contains a greater amount of useful information. The experimental results show that the D-LSTM is superior to the standard LSTM for sentences involving compositional knowledge (SICK) data.

Deconvolutional Latent-Variable Model for Text Sequence Matching

A Cross-Sentence Latent Variable Model for Semi-Supervised Text Sequence Matching

Dependency-based Siamese Long Short-Term Memory Network for Learning Sentence Representations.

Multimodal Latent Language Modeling with Next-Token Diffusion

Dual-View Variational Autoencoders for Semi-Supervised Text Matching

Fast Decoding in Sequence Models using Discrete Latent Variables

Contrastive latent variable models for neural text generation.

Language as a Latent Sequence: deep latent variable models for semi-supervised paraphrase generation

LDSeq: Latent Diffusion Models for Sequence to Sequence Text Generation

Adaptive Latent Graph Representation Learning for Image-Text Matching

Deep Fusion LSTMs for Text Semantic Matching.

Densely-Connected Transformer with Co-attentive Information for Matching Text Sequences.

On the Encoder-Decoder Incompatibility in Variational Text Modeling and Beyond

A Discriminative Latent-Variable Model for Bilingual Lexicon Induction

Convolutional Neural Network Architectures for Matching Natural Language Sentences

Improving Text Matching with Semantic Dependency Graph via Message Passing Neural Network

Improved Variational Autoencoders for Text Modeling using Dilated Convolutions

Discrete Latent Variable Representations for Low-Resource Text Classification

Latent-Glat: Glancing at Latent Variables for Parallel Text Generation

A Convolutional Deep Markov Model for Unsupervised Speech Representation Learning

A Bilingual Generative Transformer for Semantic Sentence Embedding