Abstract:Graph neural networks (GNNs) aim to learn well-trained representations in a lower-dimension space for downstream tasks while preserving the topological structures. In recent years, attention mechanism, which is brilliant in the fields of natural language processing and computer vision, is introduced to GNNs to adaptively select the discriminative features and automatically filter the noisy information. To the best of our knowledge, due to the fast-paced advances in this domain, a systematic overview of attention-based GNNs is still missing. To fill this gap, this paper aims to provide a comprehensive survey on recent advances in attention-based GNNs. Firstly, we propose a novel two-level taxonomy for attention-based GNNs from the perspective of development history and architectural perspectives. Specifically, the upper level reveals the three developmental stages of attention-based GNNs, including graph recurrent attention networks, graph attention networks, and graph transformers. The lower level focuses on various typical architectures of each stage. Secondly, we review these attention-based methods following the proposed taxonomy in detail and summarize the advantages and disadvantages of various models. A model characteristics table is also provided for a more comprehensive comparison. Thirdly, we share our thoughts on some open issues and future directions of attention-based GNNs. We hope this survey will provide researchers with an up-to-date reference regarding applications of attention-based GNNs. In addition, to cope with the rapid development in this field, we intend to share the relevant latest papers as an open resource at https://github.com/sunxiaobei/awesome-attention-based-gnns.

Graph Neural Networks for Natural Language Processing: A Survey

Graph Neural Networks for Natural Language Processing: A Survey

Graph neural networks in vision-language image understanding: a survey

Graph Neural Networks for Text Classification: A Survey

Graph Neural Networks: Taxonomy, Advances, and Trends

Graph Neural Networks: Taxonomy, Advances and Trends

A survey of graph neural networks in various learning paradigms: methods, applications, and challenges

Graph Neural Networks in Point Clouds: A Survey

Graph-level Neural Networks: Current Progress and Future Directions

A Comprehensive Survey of Graph Neural Networks for Knowledge Graphs

A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective

Deep Learning on Graphs: A Survey

A Comprehensive Survey on Graph Summarization with Graph Neural Networks

Graph Neural Networks: Methods, Applications, and Opportunities

A survey of graph neural networks and their industrial applications

A Survey of Large Language Models for Graphs

Attention-based graph neural networks: a survey