Abstract:Graph neural network (GNN) has become a popular tool to analyze the graph data. Existing GNNs only focus on networks with first-order dependency, that is, conventional networks following the Markov property. However, many networks in real life own the higher-order dependency, such as click-stream data where the choice of the next page depends not only on the current page but also on previous pages. This kind of sequential data from complex systems (including natural dependencies) are often ignored by existing GNNs which makes them ineffective. To address this problem, we propose for the first time new GNN approaches for higher-order networks in this paper. First, we form sequence fragments by the current node and its predecessor nodes of different orders as candidate higher-order dependencies. When the fragment significantly affects the probability distribution of different successor nodes of the current node, we include it in the higher-order dependency set. We formulize the network with higher-order dependency as an augmented conventional first-order network, and then feed it into GNNs to derive network embeddings. Moreover, we further propose a new end-to-end GNN framework for dealing with higher-order networks directly in the model. Specifically, the higher-order dependency is used as the neighbor aggregation controller when the node is embedded and updated. In the graph convolutional layer, in addition to the first-order neighbor information, we also aggregate the middle node information from the higher-order dependency segment. We finally test the new approaches on three real networks with higher-order dependency, and compare with some state-of-the-art methods. The results show significant improvements of the new approaches which consider higher-order dependency.

A Higher-Order Semantic Dependency Parser

High-order Graph-based Neural Dependency Parsing.

Incremental Graph-based Neural Dependency Parsing.

Neural Maximum Subgraph Parsing for Cross-Domain Semantic Dependency Analysis.

A Neural Transition-Based Approach for Semantic Dependency Graph Parsing

Dependency-Gated Cascade Biaffine Network for Chinese Semantic Dependency Graph Parsing

An Effective Neural Network Model for Graph-based Dependency Parsing.

A Semi-Autoregressive Graph Generative Model for Dependency Graph Parsing

Transition-based Dependency Parsing Using Two Heterogeneous Gated Recursive Neural Networks.

Integrative Semantic Dependency Parsing Via Efficient Large-Scale Feature Selection

Utilizing Dependency Language Models for Graph-Based Dependency Parsing Models

Global Greedy Dependency Parsing

Peking: Building Semantic Dependency Graphs with a Hybrid Parser.

Graph Neural Network for Higher-Order Dependency Networks

Transition-Based Chinese Semantic Dependency Graph Parsing.

Dependency Parsing with Dilated Iterated Graph CNNs

Probabilistic Graph-Based Dependency Parsing With Convolutional Neural Network

Graph-Based Dependency Parsing With Recursive Neural Network

Improving Semantic Parsing with Neural Generator-Reranker Architecture

Deep Hierarchical Human Semantic Parsing

Leverage External Knowledge and Self-attention for Chinese Semantic Dependency Graph Parsing