Abstract:Local Attention-guided Message Passing Mechanism (LAMP) adopted in Graph Attention Networks (GATs) is designed to adaptively learn the importance of neighboring nodes for better local aggregation on the graph, which can bring the representations of similar neighbors closer effectively, thus showing stronger discrimination ability. However, existing GATs suffer from a significant discrimination ability decline in heterophilic graphs because the high proportion of dissimilar neighbors can weaken the self-attention of the central node, jointly resulting in the deviation of the central node from similar nodes in the representation space. This kind of effect generated by neighboring nodes is called the Distraction Effect (DE) in this paper. To estimate and weaken the DE of neighboring nodes, we propose a Causally graph Attention network for Trimming heterophilic graph (CAT). To estimate the DE, since the DE are generated through two paths (grab the attention assigned to neighbors and reduce the self-attention of the central node), we use Total Effect to model DE, which is a kind of causal estimand and can be estimated from intervened data; To weaken the DE, we identify the neighbors with the highest DE (we call them Distraction Neighbors) and remove them. We adopt three representative GATs as the base model within the proposed CAT framework and conduct experiments on seven heterophilic datasets in three different sizes. Comparative experiments show that CAT can improve the node classification accuracy of all base GAT models. Ablation experiments and visualization further validate the enhancement of discrimination ability brought by CAT. The source code is available at <a class="link-external link-https" href="https://github.com/GeoX-Lab/CAT" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper aims to address the issue of decreased classification performance of Graph Attention Networks (GATs) on heterophilic graphs. Specifically, existing GATs, when dealing with heterophilic graphs, suffer from a high proportion of dissimilar neighbors that weaken the self-attention of the central node, causing the central node to deviate from its similar nodes in the representation space, thereby affecting classification accuracy. This negative impact caused by neighboring nodes is referred to as the "Distraction Effect" (DE). To estimate and mitigate DE, the authors propose a causal inference-based graph attention network trimming method (Causal Graph Attention Network for Trimming heterophilic graphs, CAT). ### Key Issues and Solutions 1. **Distraction Effect (DE)**: - **Problem**: In heterophilic graphs, a high proportion of dissimilar neighbors weakens the self-attention of the central node, causing the central node to deviate from its similar nodes, thereby affecting classification accuracy. - **Solution**: Reduce the impact of DE by identifying and removing neighbors with the highest DE (referred to as distracting neighbors). 2. **Causal Inference**: - **Problem**: How to estimate and quantify the distraction effect of neighboring nodes on the central node? - **Solution**: Use Total Effect (TE) as a metric and estimate DE by intervening in the Local Neighbor Distribution (LND). 3. **Graph Trimming**: - **Problem**: How to improve the classification performance of heterophilic graphs without modifying the GAT architecture or globally searching for new neighbors? - **Solution**: Generate a new graph by trimming (removing distracting neighbors) to improve the classification performance of GAT. ### Experimental Validation - **Experimental Setup**: The authors used three representative GAT models as baseline models and conducted experiments on seven heterophilic graph datasets of different scales. - **Experimental Results**: The experiments show that CAT can significantly improve the node classification accuracy of all baseline GAT models. Ablation experiments and visualizations further validate the enhanced discriminative ability of CAT. ### Method Characteristics - **Plug-in Framework**: CAT is a plug-in framework that can be introduced into any LAMP-based GAT, as it learns the trimmed graph during the attention learning phase without modifying the model architecture or globally searching for new neighbors. - **Theoretical Foundation**: Through causal inference and graph trimming, CAT provides a new perspective to enhance the discriminative ability of GAT in heterophilic graphs. ### Conclusion By proposing the CAT method, the paper successfully addresses the issue of decreased classification performance of GATs on heterophilic graphs. By identifying and removing distracting neighbors, CAT can improve the discriminative ability of GATs, and the method has broad applicability and flexibility.

CAT: A Causally Graph Attention Network for Trimming Heterophilic Graph

CAT: A Causal Graph Attention Network for Trimming Heterophilic Graphs

NGAT: Attention in Breadth and Depth Exploration for Semi-Supervised Graph Representation Learning

Mixed Geometry Message and Trainable Convolutional Attention Network for Knowledge Graph Completion

MC-GAT: Multi-Channel Graph Attention Networks for Capturing Diverse Information in Complex Graphs

CopGAT: Co-propagation Self-supervised Graph Attention Network

HONGAT: Graph Attention Networks in the Presence of High-Order Neighbors

LGAT: a light graph attention network focusing on message passing for semi-supervised node classification

Representation Learning on Heterophilic Graph with Directional Neighborhood Attention

Graph-CAT: Graph Co-Attention Networks Via Local and Global Attribute Augmentations

Simple and Deep Graph Attention Networks

Heterophily-Aware Graph Attention Network

Graph Context-Attention Network Via Low and High Order Aggregation

FastGAT: Simple and Efficient Graph Attention Neural Network with Global-Aware Adaptive Computational Node Attention.

KGAT: An Enhanced Graph-Based Model for Text Classification

Multimodal Heterogeneous Graph Attention Network.

Heterogeneous Graph Gated Attention Network

Community Attention Network for Semi-supervised Node Classification.

Improving Graph Attention Networks with Large Margin-based Constraints

Enhancing Heterogeneous Knowledge Graph Completion with a Novel GAT-based Approach