Abstract:How to properly model graphs is a long-existing and important problem in NLP area, where several popular types of graphs are knowledge graphs, semantic graphs and dependency graphs. Comparing with other data structures, such as sequences and trees, graphs are generally more powerful in representing complex correlations among entities. For example, a knowledge graph stores real-word entities (such as "Barack_Obama" and "U.S.") and their relations (such as "live_in" and "lead_by"). Properly encoding a knowledge graph is beneficial to user applications, such as question answering and knowledge discovery. Modeling graphs is also very challenging, probably because graphs usually contain massive and cyclic relations. Recent years have witnessed the success of deep learning, especially RNN-based models, on many NLP problems. Besides, RNNs and their variations have been extensively studied on several graph problems and showed preliminary successes. Despite the successes that have been achieved, RNN-based models suffer from several major drawbacks on graphs. First, they can only consume sequential data, thus linearization is required to serialize input graphs, resulting in the loss of important structural information. Second, the serialization results are usually very long, so it takes a long time for RNNs to encode them. In this thesis, we propose a novel graph neural network, named graph recurrent network (GRN). We study our GRN model on 4 very different tasks, such as machine reading comprehension, relation extraction and machine translation. Some take undirected graphs without edge labels, while the others have directed ones with edge labels. To consider these important differences, we gradually enhance our GRN model, such as further considering edge labels and adding an RNN decoder. Carefully designed experiments show the effectiveness of GRN on all these tasks.

Sentence Modeling with Gated Recursive Neural Network

When Are Tree Structures Necessary for Deep Learning of Representations?

Recursive Graphical Neural Networks for Text Classification

Recursive Deep Models for Discourse Parsing

A Hierarchical Model with Recurrent Convolutional Neural Networks for Sequential Sentence Classification

Dependency-based Gated Recursive Neural Network for Chinese Word Segmentation.

Hierarchically Gated Recurrent Neural Network for Sequence Modeling

TreeNet: Learning Sentence Representations with Unconstrained Tree Structure.

Transition-based Dependency Parsing Using Two Heterogeneous Gated Recursive Neural Networks.

Discriminative Neural Sentence Modeling by Tree-Based Convolution

Learning to Compose over Tree Structures via POS Tags

Graph-Based Dependency Parsing With Recursive Neural Network

Hyper-Gated Recurrent Neural Networks for Chinese Word Segmentation.

On Tree-Based Neural Sentence Modeling

Learning Tag Embeddings and Tag-specific Composition Functions in Recursive Neural Network.

Learning to Embed Sentences Using Attentive Recursive Trees.

Segment-Level Sequence Modeling Using Gated Recursive Semi-Markov Conditional Random Fields

Refined Gate: A Simple and Effective Gating Mechanism for Recurrent Units

Attention-based Recurrent Convolutional Neural Network for Automatic Essay Scoring.

Tree-based Convolution for Sentence Modeling.

Tackling Graphical NLP problems with Graph Recurrent Networks