Abstract:Textual representations play an important role in the field of natural language processing (NLP). The efficiency of NLP tasks, such as text comprehension and information extraction, can be significantly improved with proper textual representations. As neural networks are gradually applied to learn the representation of words and phrases, fairly efficient models of learning short text representations have been developed, such as the continuous bag of words (CBOW) and skip-gram models, and they have been extensively employed in a variety of NLP tasks. Because of the complex structure generated by the longer text lengths, such as sentences, algorithms appropriate for learning short textual representations are not applicable for learning long textual representations. One method of learning long textual representations is the Long Short-Term Memory (LSTM) network, which is suitable for processing sequences. However, the standard LSTM does not adequately address the primary sentence structure (subject, predicate and object), which is an important factor for producing appropriate sentence representations. To resolve this issue, this paper proposes the dependency-based LSTM model (D-LSTM). The D-LSTM divides a sentence representation into two parts: a basic component and a supporting component. The D-LSTM uses a pre-trained dependency parser to obtain the primary sentence information and generate supporting components, and it also uses a standard LSTM model to generate the basic sentence components. A weight factor that can adjust the ratio of the basic and supporting components in a sentence is introduced to generate the sentence representation. Compared with the representation learned by the standard LSTM, the sentence representation learned by the D-LSTM contains a greater amount of useful information. The experimental results show that the D-LSTM is superior to the standard LSTM for sentences involving compositional knowledge (SICK) data.

Representation Learning of Multiword Expressions with Compositionality Constraint.

Dependency-based Siamese Long Short-Term Memory Network for Learning Sentence Representations.

Contextual Compositionality Detection with External Knowledge Bases andWord Embeddings

MRSP: Learn Multi-Representations of Single Primitive for Compositional Zero-Shot Learning

Model Composition for Multimodal Large Language Models

Learning to Compose Representations of Different Encoder Layers towards Improving Compositional Generalization

Unleashing the Potentials of Likelihood Composition for Multi-modal Language Models

Meta Multi-Task Learning for Sequence Modeling.

Leveraging MLLM Embeddings and Attribute Smoothing for Compositional Zero-Shot Learning

Improving Compositional Generalization in Math Word Problem Solving

Multimodal Composition Example Mining for Composed Query Image Retrieval

Compositional Mixture Representations for Vision and Text

Measuring the Non-Compositionality of Multiword Expressions

MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models

Modeling multi-prototype Chinese word representation learning for word similarity

Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learning

Compositional Generalization in Unsupervised Compositional Representation Learning: A Study on Disentanglement and Emergent Language

Interpretable Composition Attribution Enhancement for Visio-linguistic Compositional Understanding

Automatic Extraction of Multiword Expressions Combining Statistical and Similarity Approaches

A Unified Framework for Jointly Learning Distributed Representations of Word and Attributes.

On Evaluating Multilingual Compositional Generalization with Translated Datasets