Abstract:How to properly model graphs is a long-existing and important problem in NLP area, where several popular types of graphs are knowledge graphs, semantic graphs and dependency graphs. Comparing with other data structures, such as sequences and trees, graphs are generally more powerful in representing complex correlations among entities. For example, a knowledge graph stores real-word entities (such as "Barack_Obama" and "U.S.") and their relations (such as "live_in" and "lead_by"). Properly encoding a knowledge graph is beneficial to user applications, such as question answering and knowledge discovery. Modeling graphs is also very challenging, probably because graphs usually contain massive and cyclic relations. Recent years have witnessed the success of deep learning, especially RNN-based models, on many NLP problems. Besides, RNNs and their variations have been extensively studied on several graph problems and showed preliminary successes. Despite the successes that have been achieved, RNN-based models suffer from several major drawbacks on graphs. First, they can only consume sequential data, thus linearization is required to serialize input graphs, resulting in the loss of important structural information. Second, the serialization results are usually very long, so it takes a long time for RNNs to encode them. In this thesis, we propose a novel graph neural network, named graph recurrent network (GRN). We study our GRN model on 4 very different tasks, such as machine reading comprehension, relation extraction and machine translation. Some take undirected graphs without edge labels, while the others have directed ones with edge labels. To consider these important differences, we gradually enhance our GRN model, such as further considering edge labels and adding an RNN decoder. Carefully designed experiments show the effectiveness of GRN on all these tasks.

A Generalization of Recurrent Neural Networks for Graph Embedding.

A Dilated Recurrent Neural Network-Based Model for Graph Embedding

Hierarchical recurrent neural networks for graph generation

Towards Understanding the Generalization of Graph Neural Networks.

Towards Understanding Generalization of Graph Neural Networks.

Learning Graph-Level Representations with Recurrent Neural Networks

Relation Embedding Based Graph Neural Networks for Handling Heterogeneous Graph

Enhancing Graph Neural Networks with Edge Features Through Sequential Representation

RAGraph: A General Retrieval-Augmented Graph Learning Framework

ES-GNN: Generalizing Graph Neural Networks Beyond Homophily With Edge Splitting

A data-centric framework of improving graph neural networks for knowledge graph embedding

Graph Neural Networks Inspired by Classical Iterative Algorithms

Nested Graph Neural Networks

Tackling Graphical NLP problems with Graph Recurrent Networks

GraphRARE: Reinforcement Learning Enhanced Graph Neural Network with Relative Entropy

Reinforcement Learning Enhanced Explainer for Graph Neural Networks

Heterogeneous Graph Neural Network with Hypernetworks for Knowledge Graph Embedding

Enabling Homogeneous GNNs to Handle Heterogeneous Graphs via Relation Embedding

Universal Inceptive GNNs by Eliminating the Smoothness-generalization Dilemma

Graph Neural Networks are Inherently Good Generalizers: Insights by Bridging GNNs and MLPs