Abstract:Local Attention-guided Message Passing Mechanism (LAMP) adopted in Graph Attention Networks (GATs) is designed to adaptively learn the importance of neighboring nodes for better local aggregation on the graph, which can bring the representations of similar neighbors closer effectively, thus showing stronger discrimination ability. However, existing GATs suffer from a significant discrimination ability decline in heterophilic graphs because the high proportion of dissimilar neighbors can weaken the self-attention of the central node, jointly resulting in the deviation of the central node from similar nodes in the representation space. This kind of effect generated by neighboring nodes is called the Distraction Effect (DE) in this paper. To estimate and weaken the DE of neighboring nodes, we propose a Causally graph Attention network for Trimming heterophilic graph (CAT). To estimate the DE, since the DE are generated through two paths (grab the attention assigned to neighbors and reduce the self-attention of the central node), we use Total Effect to model DE, which is a kind of causal estimand and can be estimated from intervened data; To weaken the DE, we identify the neighbors with the highest DE (we call them Distraction Neighbors) and remove them. We adopt three representative GATs as the base model within the proposed CAT framework and conduct experiments on seven heterophilic datasets in three different sizes. Comparative experiments show that CAT can improve the node classification accuracy of all base GAT models. Ablation experiments and visualization further validate the enhancement of discrimination ability brought by CAT. The source code is available at <a class="link-external link-https" href="https://github.com/GeoX-Lab/CAT" rel="external noopener nofollow">this https URL</a>.

HONGAT: Graph Attention Networks in the Presence of High-Order Neighbors

NGAT: Attention in Breadth and Depth Exploration for Semi-Supervised Graph Representation Learning

Neighbor Overlay-Induced Graph Attention Network

High-order graph attention network

Graph Context-Attention Network Via Low and High Order Aggregation

Hybrid High-order in Graph Attention Layer.

Multi-head multi-order graph attention networks

Not All Neighbors Are Worth Attending To: Graph Selective Attention Networks for Semi-supervised Learning

FastGAT: Simple and Efficient Graph Attention Neural Network with Global-Aware Adaptive Computational Node Attention.

How Attentive are Graph Attention Networks?

Simple and Deep Graph Attention Networks

Representation Learning on Heterophilic Graph with Directional Neighborhood Attention

Adaptive Depth Graph Attention Networks

MC-GAT: Multi-Channel Graph Attention Networks for Capturing Diverse Information in Complex Graphs

CAT: A Causally Graph Attention Network for Trimming Heterophilic Graph

Graph Joint Attention Networks

Graph Ordering Attention Networks

CAT: A Causal Graph Attention Network for Trimming Heterophilic Graphs

Graph-CAT: Graph Co-Attention Networks Via Local and Global Attribute Augmentations

Ha-gnn: a novel graph neural network based on hyperbolic attention

Graph Oriented Attention Networks