Abstract:Recursive neural models, which use syntactic parse trees to recursively generate representations bottom-up, are a popular architecture. However there have not been rigorous evaluations showing for exactly which tasks this syntax-based method is appropriate. In this paper, we benchmark recursive neural models against sequential recurrent neural models, enforcing applesto-apples comparison as much as possible. We investigate 4 tasks: (1) sentiment classification at the sentence level and phrase level; (2) matching questions to answerphrases; (3) discourse parsing; (4) semantic relation extraction. Our goal is to understand better when, and why, recursive models can outperform simpler models. We find that recursive models help mainly on tasks (like semantic relation extraction) that require longdistance connection modeling, particularly on very long sequences. We then introduce a method for allowing recurrent models to achieve similar performance: breaking long sentences into clause-like units at punctuation and processing them separately before combining. Our results thus help understand the limitations of both classes of models, and suggest directions for improving recurrent models.

A Re-ranking Model for Dependency Parser with Recursive Convolutional Neural Network

When Are Tree Structures Necessary for Deep Learning of Representations?

Graph-Based Dependency Parsing With Recursive Neural Network

A Search-Based Dynamic Reranking Model for Dependency Parsing.

A Neural Network Based Translation Constrained Reranking Model for Chinese Dependency Parsing

Recursive Deep Models for Discourse Parsing

Discriminative Parse Reranking for Chinese with Homogeneous and Heterogeneous Annotations

Ranking With Recursive Neural Networks And Its Application To Multi-Document Summarization

Applying Tree Kernel to Chinese Syntactic Parsing Reranking

A Hierarchical Model with Recurrent Convolutional Neural Networks for Sequential Sentence Classification

Transition-based Dependency Parsing Using Two Heterogeneous Gated Recursive Neural Networks.

Adaptive Multi-Compositionality for Recursive Neural Network Models

Global discriminative model for dependency parsing in NLP pipeline

An Effective Neural Network Model for Graph-based Dependency Parsing.

Learning to Compose over Tree Structures via POS Tags

Learning Tag Embeddings and Tag-specific Composition Functions in Recursive Neural Network.

Dependency-based Convolutional Neural Networks for Sentence Embedding

Sentence Modeling with Gated Recursive Neural Network

Dynamic Compositional Neural Networks over Tree Structure

Dependency Sensitive Convolutional Neural Networks for Modeling Sentences and Documents

A Position Encoding Convolutional Neural Network Based on Dependency Tree for Relation Classification