Abstract:Graph Neural Networks (GNNs) have become one of the indispensable tools to learn from graph-structured data, and their usefulness has been shown in wide variety of tasks. In recent years, there have been tremendous improvements in architecture design, resulting in better performance on various prediction tasks. In general, these neural architectures combine node feature aggregation and feature transformation using learnable weight matrix in the same layer. This makes it challenging to analyze the importance of node features aggregated from various hops and the expressiveness of the neural network layers. As different graph datasets show varying levels of homophily and heterophily in features and class label distribution, it becomes essential to understand which features are important for the prediction tasks without any prior information. In this work, we decouple the node feature aggregation step and depth of graph neural network, and empirically analyze how different aggregated features play a role in prediction performance. We show that not all features generated via aggregation steps are useful, and often using these less informative features can be detrimental to the performance of the GNN model. Through our experiments, we show that learning certain subsets of these features can lead to better performance on wide variety of datasets. Based on our observations, we introduce several key design strategies for graph neural networks. More specifically, we propose to use softmax as a regularizer and ”soft-selector” of features aggregated from neighbors at different hop distances; and L2-Normalization over GNN layers. Combining these techniques, we present a simple and shallow model, Feature Selection Graph Neural Network (FSGNN), and show empirically that the proposed model achieves comparable or even higher accuracy than state-of-the-art GNN models in nine benchmark datasets for the node classification task, with remarkable improvements up to 51.1%. Source code available at https://github.com/sunilkmaurya/FSGNN/.

Simple GNNs with Low Rank Non-parametric Aggregators

Simplifying GNN Performance with Low Rank Kernel Models

NGAT: Attention in Breadth and Depth Exploration for Semi-Supervised Graph Representation Learning

Deep Graph Neural Networks via Flexible Subgraph Aggregation

Global Minima, Recoverability Thresholds, and Higher-Order Structure in GNNS

Higher-Order GNNs Meet Efficiency: Sparse Sobolev Graph Neural Networks

Simplifying approach to node classification in Graph Neural Networks

From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness

GAIN: Graph Attention & Interaction Network for Inductive Semi-Supervised Learning over Large-scale Graphs

Non-negative low rank and sparse graph for semi-supervised learning

Graph Stochastic Neural Networks for Semi-supervised Learning.

Going Deep: Graph Convolutional Ladder-Shape Networks

Graph Neural Networks Are More Than Filters: Revisiting and Benchmarking from A Spectral Perspective

A Unified View on Graph Neural Networks as Graph Signal Denoising

Graph Sampling for Scalable and Expressive Graph Neural Networks on Homophilic Graphs

Improving the interpretability of GNN predictions through conformal-based graph sparsification

Dual GNNs: Graph Neural Network Learning with Limited Supervision

Learning Graph Neural Networks with Positive and Unlabeled Nodes

Graph Entropy Minimization for Semi-supervised Node Classification

E2GNN: Efficient Graph Neural Network Ensembles for Semi-Supervised Classification

Graph Neural Networks Inspired by Classical Iterative Algorithms