Abstract:To improve the performance of Graph Neural Networks (GNNs), Graph Structure Learning (GSL) has been extensively applied to reconstruct or refine original graph structures, effectively addressing issues like heterophily, over-squashing, and noisy structures. While GSL is generally thought to improve GNN performance, it often leads to longer training times and more hyperparameter tuning. Besides, the distinctions among current GSL methods remain ambiguous from the perspective of GNN training, and there is a lack of theoretical analysis to quantify their effectiveness. Recent studies further suggest that, under fair comparisons with the same hyperparameter tuning, GSL does not consistently outperform baseline GNNs. This motivates us to ask a critical question: is GSL really useful for GNNs? To address this question, this paper makes two key contributions. First, we propose a new GSL framework, which includes three steps: GSL base (the representation used for GSL) construction, new structure construction, and view fusion, to better understand the effectiveness of GSL in GNNs. Second, after graph convolution, we analyze the differences in mutual information (MI) between node representations derived from the original topology and those from the newly constructed topology. Surprisingly, our empirical observations and theoretical analysis show that no matter which type of graph structure construction methods are used, after feeding the same GSL bases to the newly constructed graph, there is no MI gain compared to the original GSL bases. To fairly reassess the effectiveness of GSL, we conduct ablation experiments and find that it is the pretrained GSL bases that enhance GNN performance, and in most cases, GSL cannot improve GNN performance. This finding encourages us to rethink the essential components in GNNs, such as self-training and structural encoding, in GNN design rather than GSL.

The Geometric Structure of Generalized Softmax Learning

A Study of Neural Collapse Phenomenon: Grassmannian Frame, Symmetry and Generalization

Exploring the Frontiers of Softmax: Provable Optimization, Applications in Diffusion Model, and Beyond

Semantic Softmax Loss for Zero-Shot Learning

Attention Scheme Inspired Softmax Regression

An ensemble learning interpretation of geometric semantic genetic programming

Stop-Gradient Softmax Loss for Deep Metric Learning.

Neural Gaussian Similarity Modeling for Differential Graph Structure Learning

Gaussian Universality in Neural Network Dynamics with Generalized Structured Input Distributions

On Geometric Structure of Activation Spaces in Neural Networks

GSSF: Generalized Structural Sparse Function for Deep Cross-modal Metric Learning

SE-GSL: A General and Effective Graph Structure Learning Framework Through Structural Entropy Optimization

$\mathcal{G}$-softmax: Improving Intra-class Compactness and Inter-class Separability of Features

Rethinking Structure Learning For Graph Neural Networks

The Performance Analysis of Generalized Margin Maximizer (GMM) on Separable Data

Generalized Single-Hidden Layer Feedforward Networks

A Unified Scheme of ResNet and Softmax

A Global Geometric Analysis of Maximal Coding Rate Reduction

On the Expressive Power of Geometric Graph Neural Networks

On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning.

Compact Graph Structure Learning via Mutual Information Compression