Spectral Graph Attention Network with Fast Eigen-approximation

Heng Chang,Yu Rong,Tingyang Xu,Wenbing Huang,Somayeh Sojoudi,Junzhou Huang,Wenwu Zhu

DOI: https://doi.org/10.48550/arXiv.2003.07450

2021-07-27

Abstract:Variants of Graph Neural Networks (GNNs) for representation learning have been proposed recently and achieved fruitful results in various fields. Among them, Graph Attention Network (GAT) first employs a self-attention strategy to learn attention weights for each edge in the spatial domain. However, learning the attentions over edges can only focus on the local information of graphs and greatly increases the computational costs. In this paper, we first introduce the attention mechanism in the spectral domain of graphs and present Spectral Graph Attention Network (SpGAT) that learns representations for different frequency components regarding weighted filters and graph wavelets bases. In this way, SpGAT can better capture global patterns of graphs in an efficient manner with much fewer learned parameters than that of GAT. Further, to reduce the computational cost of SpGAT brought by the eigen-decomposition, we propose a fast approximation variant SpGAT-Cheby. We thoroughly evaluate the performance of SpGAT and SpGAT-Cheby in semi-supervised node classification tasks and verify the effectiveness of the learned attentions in the spectral domain.

Machine Learning,Social and Information Networks

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the limitations of existing graph neural networks (GNNs) in processing graph - structured data. In particular, the Graph Attention Network (GAT) can only focus on the local information of the graph and has a relatively high computational cost. To overcome these limitations, the paper proposes the Spectral Graph Attention Network (SpGAT), which can more effectively capture the global patterns of the graph by introducing the attention mechanism into the spectral domain of the graph and uses fewer learning parameters. In addition, in order to further reduce the computational cost of SpGAT due to eigen - decomposition, the paper also proposes a fast approximate variant SpGAT - Cheby, which utilizes Chebyshev polynomial approximation to accelerate the calculation of spectral graph wavelets.

Specifically, SpGAT selects graph wavelets as the spectral basis and decomposes them into low - frequency and high - frequency components. Then it constructs two different convolution kernels respectively and applies the attention mechanism on these two kernels to capture their importance. This method can not only explicitly encode the structural information of the graph from a global perspective, but also optimize the model performance by controlling the proportion of low - frequency and high - frequency components. The experimental results show that SpGAT and its fast approximate variant SpGAT - Cheby have achieved state - of - the - art results in the semi - supervised node classification task, which verifies the effectiveness of the spectral - domain attention mechanism.