Abstract:Graph neural networks (GNNs) have been highly successful in graph representation learning. The goal of GNNs is to enrich node representations by aggregating information from neighboring nodes. Much work has attempted to improve the quality of aggregation by introducing a variety of graph information with representational capabilities. The class of GNNs that improves the quality of aggregation by encoding graph information with representational capabilities into the weights of neighboring nodes through different learnable transformation structures (LTSs) are referred to as implicit GNNs. However, we argue that LTSs only transform graph information into the weights of neighboring nodes in the direction that minimizes the loss function during the learning process and does not actually utilize the effective properties of graph information, a phenomenon that we refer to as graph information vanishing (GIV). To validate this point, we perform thousands of experiments on seven node classification benchmark datasets. We first replace the graph information utilized by five implicit GNNs with random values and surprisingly observe that the variation range of accuracies is less than ± 0.3%. Then, we quantitatively characterize the similarity of the weights generated from graph information and random values by cosine similarity, and the cosine similarities are greater than 0.99. The empirical experiments show that graph information is equivalent to initializing the input of LTSs. We believe that graph information as an additional supervised signal to constrain the training of GNNs can effectively solve GIV. Here, we propose GinfoNN, which utilizes both labels and discrete graph curvature as supervised signals to jointly constrain the training of the model. The experimental results show that the classification accuracies of GinfoNN improve by two percentage points over baselines on large and dense datasets.

Variational Inference for Training Graph Neural Networks in Low-Data Regime Through Joint Structure-Label Estimation

Graph Stochastic Neural Networks for Semi-supervised Learning.

Hierarchical Graph Neural Network Based-on Semi-implicit Variational Inference

Learning Strong Graph Neural Networks with Weak Information

Self-supervised Graph Representations with Generative Adversarial Learning

Robust Node Representation Learning via Graph Variational Diffusion Networks

Utilizing Edge Features in Graph Neural Networks Via Variational Information Maximization

Graph Structure Learning with Variational Information Bottleneck

Substructure Aware Graph Neural Networks.

Effective Stabilized Self-Training on Few-Labeled Graph Data

Generalized Equivariance and Preferential Labeling for GNN Node Classification

Wide Graph Neural Network

Multiview Subgraph Neural Networks: Self-Supervised Learning With Scarce Labeled Data

Imbalanced Graph Classification via Graph-of-Graph Neural Networks

Variational Gridded Graph Convolution Network for Node Classification

Graph Information Vanishing Phenomenon in Implicit Graph Neural Networks

Dual GNNs: Graph Neural Network Learning with Limited Supervision

Learning on Graphs with Out-of-Distribution Nodes

Learning graph-level representation from local-structural distribution with Graph Neural Networks

Semisupervised Graph Neural Networks for Graph Classification

Semi-Supervised Mixture Learning for Graph Neural Networks With Neighbor Dependence