Abstract:Traditional graph neural networks (GNNs) lack scalability and lose individual node characteristics due to over-smoothing, especially in the case of deeper networks. This results in sub-optimal feature representation, affecting the model's performance on tasks involving dynamically changing graphs. To address this issue, we present Graph Selective States Focused Attention Networks (GSANs) based neural network architecture for graph-structured data. The GSAN is enabled by multi-head masked self-attention (MHMSA) and selective state space modeling (S3M) layers to overcome the limitations of GNNs. In GSAN, the MHMSA allows GSAN to dynamically emphasize crucial node connections, particularly in evolving graph environments. The S3M layer enables the network to adjust dynamically in changing node states and improving predictions of node behavior in varying contexts without needing primary knowledge of the graph structure. Furthermore, the S3M layer enhances the generalization of unseen structures and interprets how node states influence link importance. With this, GSAN effectively outperforms inductive and transductive tasks and overcomes the issues that traditional GNNs experience. To analyze the performance behavior of GSAN, a set of state-of-the-art comparative experiments are conducted on graphs benchmark datasets, including $Cora$, $Citeseer$, $Pubmed$ network citation, and $protein-protein-interaction$ datasets, as an outcome, GSAN improved the classification accuracy by $1.56\%$, $8.94\%$, $0.37\%$, and $1.54\%$ on $F1-score$ respectively.

Sparse Graphs-based Dynamic Attention Networks

NGAT: Attention in Breadth and Depth Exploration for Semi-Supervised Graph Representation Learning

SPAN: Subgraph Prediction Attention Network for Dynamic Graphs

Focus Where It Matters: Graph Selective State Focused Attention Networks

How Attentive are Graph Attention Networks?

Adaptive Structural Fingerprints for Graph Attention Networks.

Towards Deep Attention in Graph Neural Networks: Problems and Remedies

FastGAT: Simple and Efficient Graph Attention Neural Network with Global-Aware Adaptive Computational Node Attention.

Spiking GATs: Learning Graph Attentions via Spiking Neural Network

SPAGAN: Shortest Path Graph Attention Network.

Revisiting Attention-Based Graph Neural Networks for Graph Classification

Simple and Deep Graph Attention Networks

A Unified Deep Sparse Graph Attention Network for Scene Graph Generation

A Spatial-Temporal Graph Mining Algorithm Based on Spatial-Temporal Sparse Attention

High-order graph attention network

Not All Neighbors Are Worth Attending To: Graph Selective Attention Networks for Semi-supervised Learning

Enhancing Graph Representation Learning with Attention-Driven Spiking Neural Networks

SDG: A Simplified and Dynamic Graph Neural Network

GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs

Attention-based Graph Neural Network for Semi-supervised Learning

Adaptive Depth Graph Attention Networks