Abstract:In this paper, we propose a sequential neural encoder with latent structured description (SNELSD) for modeling sentences. This model introduces latent chunk-level representations into conventional sequential neural encoders, i.e., recurrent neural networks with long short-term memory (LSTM) units, to consider the compositionality of languages in semantic modeling. An SNELSD model has a hierarchical structure that includes a detection layer and a description layer. The detection layer predicts the boundaries of latent word chunks in an input sentence and derives a chunk-level vector for each word. The description layer utilizes modified LSTM units to process these chunk-level vectors in a recurrent manner and produces sequential encoding outputs. These output vectors are further concatenated with word vectors or the outputs of a chain LSTM encoder to obtain the final sentence representation. All the model parameters are learned in an end-to-end manner without a dependency on additional text chunking or syntax parsing. A natural language inference task and a sentiment analysis task are adopted to evaluate the performance of our proposed model. The experimental results demonstrate the effectiveness of the proposed SNELSD model on exploring task-dependent chunking patterns during the semantic modeling of sentences. Furthermore, the proposed method achieves better performance than conventional chain LSTMs and tree-structured LSTMs on both tasks.

A Hierarchical Model with Recurrent Convolutional Neural Networks for Sequential Sentence Classification

Hierarchical Neural Networks for Sequential Sentence Classification in Medical Scientific Abstracts

Residual Recurrent Neural Networks for Learning Sequential Representations.

Sliding Hierarchical Recurrent Neural Networks for Sequence Classification

Dependency-based Convolutional Neural Networks for Sentence Embedding

Attention-based Recurrent Convolutional Neural Network for Automatic Essay Scoring.

LSTM-based Deep Neural Network With A Focus on Sentence Representation for Sequential Sentence Classification in Medical Scientific Abstracts

Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks

Sentence Modeling with Gated Recursive Neural Network

Dependency Sensitive Convolutional Neural Networks for Modeling Sentences and Documents

Tree-based Convolution for Sentence Modeling.

Chinese Semantic Role Labeling with Bidirectional Recurrent Neural Networks

Focused Hierarchical RNNs for Conditional Sequence Processing

Learning text representation using recurrent convolutional neural network with highway layers

A Deep Neural Network Sentence Level Classification Method with Context Information

Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks

Contextual Recurrent Units for Cloze-style Reading Comprehension

A Sequential Neural Encoder with Latent Structured Description for Modeling Sentences.

Hierarchical Neural Network with Serial Attention Mechanism for Review Sentiment Classifification

A C-LSTM Neural Network for Text Classification

A Sequential Model for Classifying Temporal Relations between Intra-Sentence Events