Abstract:Textual representations play an important role in the field of natural language processing (NLP). The efficiency of NLP tasks, such as text comprehension and information extraction, can be significantly improved with proper textual representations. As neural networks are gradually applied to learn the representation of words and phrases, fairly efficient models of learning short text representations have been developed, such as the continuous bag of words (CBOW) and skip-gram models, and they have been extensively employed in a variety of NLP tasks. Because of the complex structure generated by the longer text lengths, such as sentences, algorithms appropriate for learning short textual representations are not applicable for learning long textual representations. One method of learning long textual representations is the Long Short-Term Memory (LSTM) network, which is suitable for processing sequences. However, the standard LSTM does not adequately address the primary sentence structure (subject, predicate and object), which is an important factor for producing appropriate sentence representations. To resolve this issue, this paper proposes the dependency-based LSTM model (D-LSTM). The D-LSTM divides a sentence representation into two parts: a basic component and a supporting component. The D-LSTM uses a pre-trained dependency parser to obtain the primary sentence information and generate supporting components, and it also uses a standard LSTM model to generate the basic sentence components. A weight factor that can adjust the ratio of the basic and supporting components in a sentence is introduced to generate the sentence representation. Compared with the representation learned by the standard LSTM, the sentence representation learned by the D-LSTM contains a greater amount of useful information. The experimental results show that the D-LSTM is superior to the standard LSTM for sentences involving compositional knowledge (SICK) data.

Attentive Siamese LSTM Network for Semantic Textual Similarity Measure

Dependency-based Siamese Long Short-Term Memory Network for Learning Sentence Representations.

Fine-grained Semantic Textual Similarity Measurement Via a Feature Separation Network

Chinese Sentences Similarity via Cross-Attention Based Siamese Network

Siamese Recurrent Architectures for Learning Sentence Similarity

Computing Semantic Text Similarity Using Rich Features.

Hybrid Attention Based Neural Architecture for Text Semantics Similarity Measurement

Re-LSTM: A long short-term memory network text similarity algorithm based on weighted word embedding

Sentence-Embedding and Similarity via Hybrid Bidirectional-LSTM and CNN Utilizing Weighted-Pooling Attention

Improving Multilingual Semantic Textual Similarity with Shared Sentence Encoder for Low-resource Languages

STSG: A Short Text Semantic Graph Model for Similarity Computing Based on Dependency Parsing and Pre-trained Language Models

A novel locality-sensitive hashing relational graph matching network for semantic textual similarity measurement

Similarity-based Text Recognition by Deeply Supervised Siamese Network

Traditional Chinese Medicine Text Similarity Calculation Model Based on the Bidirectional Temporal Siamese Network

Learning Semantic Textual Similarity from Conversations

An Adversarial Joint Learning Model for Low-Resource Language Semantic Textual Similarity.

C-STS: Conditional Semantic Textual Similarity

A hybrid approach of Poisson distribution LDA with deep Siamese Bi-LSTM and GRU model for semantic similarity prediction for text data

Measurement of Semantic Textual Similarity in Clinical Texts: Comparison of Transformer-Based Models

ECNUCS: Measuring Short Text Semantic Equivalence Using Multiple Similarity Measurements.

Hybridization between Neural Computing and Nature-Inspired Algorithms for a Sentence Similarity Model Based on the Attention Mechanism