Abstract:In recent years, graph convolutional neural networks (GCNs) have become a popular research topic due to their outstanding performance in various complex network data mining tasks. However, current research on graph neural networks lacks understanding of the high-order structural features of networks, focusing mostly on node features and first-order neighbor features. This article proposes two new models, MGAT and MGATv2, by introducing high-order structure motifs that frequently appear in networks and combining them with graph attention mechanisms. By introducing a mixed information matrix based on motifs, the generation process of graph attention coefficients is improved, allowing the model to capture higher-order structural features. Compared with the latest research on various graph neural networks, both MGAT and MGATv2 achieve good results in node classification tasks. Furthermore, through various experimental studies on real datasets, we demonstrate that the introduction of network structural motifs can effectively enhance the expressive power of graph neural networks, indicating that both high-order structural features and attribute features are important components of network feature learning.

What problem does this paper attempt to address?

The paper primarily aims to address the issues faced by Graph Neural Networks (GNNs) when dealing with complex network data, particularly the insufficient attention to higher-order structural features in current research. Specifically, the paper proposes two new models—MGAT (Motif-based Graph Attention Networks) and MGATv2, which enhance the graph attention mechanism's ability to learn these features by introducing higher-order structural features (i.e., frequently occurring subgraph structures, called "motifs") into the network. The paper points out that existing Graph Convolutional Networks (GCNs) mainly focus on node attribute features and first-order neighbor features, while ignoring higher-order structural features. To improve this, the proposed method combines graph attention mechanisms with higher-order structural motifs, designing new models capable of capturing higher-order structural information of the graph. Specifically: 1. **Research Motivation**: The research is based on an observation that existing methods often only consider the direct neighbors of nodes (i.e., first-order neighbors), which may lead to models not fully utilizing global features and being prone to overfitting. 2. **Proposed Models**: - **MGAT**: By constructing a motif-based hybrid information matrix (combining the original adjacency matrix and the motif-based adjacency matrix), the generation process of graph attention coefficients is improved. This allows for better capturing of higher-order structural features of the graph. - **MGATv2**: Based on the dynamic attention mechanism of GATv2, it also introduces a motif-based hybrid information matrix, further enhancing the model's ability to capture higher-order structural features. 3. **Experimental Validation**: The paper conducts extensive experimental validation on multiple real-world datasets, including node classification tasks, hyperparameter analysis experiments, and robustness experiments, to demonstrate the effectiveness of the proposed methods. In summary, the main contribution of this paper is the proposal of a new graph attention network model that can effectively utilize higher-order structural features of the graph, and experimental results demonstrate its superior performance in tasks such as node classification.

MGATs: Motif-Based Graph Attention Networks

NGAT: Attention in Breadth and Depth Exploration for Semi-Supervised Graph Representation Learning

Mixed Geometry Message and Trainable Convolutional Attention Network for Knowledge Graph Completion

High-order graph attention network

Motif Graph Neural Network

Multi-head multi-order graph attention networks

MMGAT: a graph attention network framework for ATAC-seq motifs finding

Representation learning of graphs using graph convolutional multilayer networks based on motifs

Meta-structure-based graph attention networks

Motif-induced Graph Normalization

Adaptive Structural Fingerprints for Graph Attention Networks.

Revisiting Attention-Based Graph Neural Networks for Graph Classification

Edge-Featured Graph Attention Network

HONGAT: Graph Attention Networks in the Presence of High-Order Neighbors

Adaptive Depth Graph Attention Networks

Graph Oriented Attention Networks

Graph Adaptive Attention Network with Cross-Entropy

A gated graph attention network based on dual graph convolution for node embedding

LGAT: a light graph attention network focusing on message passing for semi-supervised node classification

Heterogeneous Graph Attention Network with Motif Clique

Aspect-Aware Graph Attention Network for Heterogeneous Information Networks