Abstract:Graph machine learning has enjoyed a meteoric rise in popularity since the introduction of deep learning in graph contexts. This is no surprise due to the ubiquity of graph data in large scale industrial settings. Tacitly assumed in all graph learning tasks is the separation of the graph structure and node features: node features strictly encode individual data while the graph structure consists only of pairwise interactions. The driving belief is that node features are (by themselves) insufficient for these tasks, so benchmark performance accurately reflects improvements in graph learning. In our paper, we challenge this orthodoxy by showing that, surprisingly, node features are oftentimes more-than-sufficient for many common graph benchmarks, breaking this critical assumption. When comparing against a well-tuned feature-only MLP baseline on seven of the most commonly used graph learning datasets, one gains little benefit from using graph structure on five datasets. We posit that these datasets do not benefit considerably from graph learning because the features themselves already contain enough graph information to obviate or substantially reduce the need for the graph. To illustrate this point, we perform a feature study on these datasets and show how the features are responsible for closing the gap between MLP and graph-method performance. Further, in service of introducing better empirical measures of progress for graph neural networks, we present a challenging parametric family of principled synthetic datasets that necessitate graph information for nontrivial performance. Lastly, we section out a subset of real-world datasets that are not trivially solved by an MLP and hence serve as reasonable benchmarks for graph neural networks.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to challenge and re - evaluate a widely - accepted assumption in graph learning: that the graph structure is necessary for graph - learning tasks. Specifically, through experiments and analysis, the authors show that on many commonly - used graph benchmark datasets, a multi - layer perceptron (MLP) using only node features can achieve performance comparable to or even better than that of graph neural networks (GNNs). This implies that the node features in these datasets already contain sufficient graph information, making the graph structure not indispensable in these tasks. #### Main research questions include: 1. **Questioning the necessity of the graph structure**: - The core problem of the paper is to explore whether the graph structure is really essential in certain graph - learning tasks. Through experiments on seven commonly - used graph datasets, the authors find that on five of these datasets, a well - tuned MLP model can significantly narrow the performance gap with graph neural networks and even outperform them in some cases. 2. **Analyzing the role of node features**: - The authors further analyze the role of node features in these datasets. They show through feature study how node features "leak" graph - structure information, thus reducing the need for an explicit graph structure. For example, on the Amazon Computers dataset, as the number of features increases, the performance of the MLP gradually approaches and even exceeds that of the GCN. 3. **Proposing new benchmark datasets**: - In order to better measure the effectiveness of graph neural networks, the authors design a class of synthetic Watts - Strogatz graph datasets. These datasets are characterized by the need to use the graph structure to obtain non - trivial performance improvements, thus providing a more challenging benchmark - testing environment for graph - learning methods. 4. **Selecting appropriate real - world datasets**: - Finally, the authors also point out some real - world datasets (such as Cora, CiteSeer), which cannot be easily solved by a simple MLP and can therefore be used as more reasonable benchmark - testing datasets for graph neural networks. ### Summary The main contribution of this paper lies in revealing the possible limitations of existing graph - learning benchmark datasets and proposing more stringent benchmark - testing methods to ensure that future research can more accurately evaluate the actual effects of graph neural networks. By introducing new synthetic datasets and selecting appropriate real - world datasets, the authors provide a more rigorous research framework for the field of graph learning.

Revisiting the Necessity of Graph Learning and Common Graph Benchmarks

Rethinking the Effectiveness of Graph Classification Datasets in Benchmarks for Assessing GNNs

Addressing Shortcomings in Fair Graph Learning Datasets: Towards a New Benchmark

Taxonomy of Benchmarks in Graph Representation Learning

State of the Art and Potentialities of Graph-level Learning

A Fair Comparison of Graph Neural Networks for Graph Classification

Shedding Light on Problems with Hyperbolic Graph Learning

UGSL: A Unified Framework for Benchmarking Graph Structure Learning

Graph Neural Networks Are More Than Filters: Revisiting and Benchmarking from A Spectral Perspective

Hyperbolic Benchmarking Unveils Network Topology-Feature Relationship in GNN Performance

TabGraphs: A Benchmark and Strong Baselines for Learning on Graphs with Tabular Node Features

A survey of graph neural networks in various learning paradigms: methods, applications, and challenges

Foundations and Frontiers of Graph Learning Theory

Graph Generative Model for Benchmarking Graph Neural Networks

Machine Learning on Graphs: A Model and Comprehensive Taxonomy

Graph Neural Networks: Methods, Applications, and Opportunities

Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models

NeuroGraph: Benchmarks for Graph Machine Learning in Brain Connectomics

Data‐efficient graph learning: Problems, progress, and prospects

NAS-Bench-Graph: Benchmarking Graph Neural Architecture Search

Deep learning for dynamic graphs: models and benchmarks