Abstract:Textual representations play an important role in the field of natural language processing (NLP). The efficiency of NLP tasks, such as text comprehension and information extraction, can be significantly improved with proper textual representations. As neural networks are gradually applied to learn the representation of words and phrases, fairly efficient models of learning short text representations have been developed, such as the continuous bag of words (CBOW) and skip-gram models, and they have been extensively employed in a variety of NLP tasks. Because of the complex structure generated by the longer text lengths, such as sentences, algorithms appropriate for learning short textual representations are not applicable for learning long textual representations. One method of learning long textual representations is the Long Short-Term Memory (LSTM) network, which is suitable for processing sequences. However, the standard LSTM does not adequately address the primary sentence structure (subject, predicate and object), which is an important factor for producing appropriate sentence representations. To resolve this issue, this paper proposes the dependency-based LSTM model (D-LSTM). The D-LSTM divides a sentence representation into two parts: a basic component and a supporting component. The D-LSTM uses a pre-trained dependency parser to obtain the primary sentence information and generate supporting components, and it also uses a standard LSTM model to generate the basic sentence components. A weight factor that can adjust the ratio of the basic and supporting components in a sentence is introduced to generate the sentence representation. Compared with the representation learned by the standard LSTM, the sentence representation learned by the D-LSTM contains a greater amount of useful information. The experimental results show that the D-LSTM is superior to the standard LSTM for sentences involving compositional knowledge (SICK) data.

Bi-directional long short term memory-gated recurrent unit model for Amharic next word prediction.

Dependency-based Siamese Long Short-Term Memory Network for Learning Sentence Representations.

Enhancing Bangla Language Next Word Prediction and Sentence Completion through Extended RNN with Bi-LSTM Model On N-gram Language

On extended long short-term memory and dependent bidirectional recurrent neural network

Handwritten Amharic Character Recognition Using a Convolutional Neural Network

Next Word Prediction Using Deep Learning: A Comparative Study

Improving Amharic Handwritten Word Recognition Using Auxiliary Task

Gated recurrent unit predictor model-based adaptive differential pulse code modulation speech decoder

Handwritten Amharic Word Recognition With Additive Attention Mechanism

Attention-based Memory Selection Recurrent Network for Language Modeling

Text prediction recurrent neural networks using long short-term memory-dropout

Context based machine translation with recurrent neural network for English–Amharic translation

English to Bangla Machine Translation Using Recurrent Neural Network

Neural Machine Translation with Word Predictions.

Predicting Next Application Most Likely Used with Word Embedding and Time-Series Data Encoding

Recurrent Memory Networks for Language Modeling

Introducing various Semantic Models for Amharic: Experimentation and Evaluation with multiple Tasks and Datasets

Bidirectional Encoder–Decoder Model for Arabic Named Entity Recognition

NEWLSTM: an Optimized Long Short-Term Memory Language Model for Sequence Prediction.

Automated Word Prediction in Bangla Language Using Stochastic Language Models

Next word prediction based on the N-gram model for Kurdish Sorani and Kurmanji