Abstract:Graph neural networks (GNNs) are widely used in domains like social networks and biological systems. However, the locality assumption of GNNs, which limits information exchange to neighboring nodes, hampers their ability to capture long-range dependencies and global patterns in graphs. To address this, we propose a new inductive bias based on variational analysis, drawing inspiration from the Brachistochrone problem. Our framework establishes a mapping between discrete GNN models and continuous diffusion functionals. This enables the design of application-specific objective functions in the continuous domain and the construction of discrete deep models with mathematical guarantees. To tackle over-smoothing in GNNs, we analyze the existing layer-by-layer graph embedding models and identify that they are equivalent to l2-norm integral functionals of graph gradients, which cause over-smoothing. Similar to edge-preserving filters in image denoising, we introduce total variation (TV) to align the graph diffusion pattern with global community topologies. Additionally, we devise a selective mechanism to address the trade-off between model depth and over-smoothing, which can be easily integrated into existing GNNs. Furthermore, we propose a novel generative adversarial network (GAN) that predicts spreading flows in graphs through a neural transport equation. To mitigate vanishing flows, we customize the objective function to minimize transportation within each community while maximizing inter-community flows. Our GNN models achieve state-of-the-art (SOTA) performance on popular graph learning benchmarks such as Cora, Citeseer, and Pubmed.

Revisiting Over-smoothing and Over-squashing using Ollivier's Ricci Curvature

Understanding over-squashing and bottlenecks on graphs via curvature

CurvDrop: A Ricci Curvature Based Approach to Prevent Graph Neural Networks from Over-Smoothing and Over-Squashing.

Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization

On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks

The Effectiveness of Curvature-Based Rewiring and the Role of Hyperparameters in GNNs Revisited

DeepRicci: Self-supervised Graph Structure-Feature Co-Refinement for Alleviating Over-squashing

Oversquashing in GNNs through the lens of information contraction and graph expansion

Discovering the Representation Bottleneck of Graph Neural Networks from Multi-order Interactions

How Curvature Enhance the Adaptation Power of Framelet GCNs

Over-Squashing in Riemannian Graph Neural Networks

Rewiring Techniques to Mitigate Oversquashing and Oversmoothing in GNNs: A Survey

A Fractional Graph Laplacian Approach to Oversmoothing

A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networks

A Survey on Oversmoothing in Graph Neural Networks

Rewiring Networks for Graph Neural Network Training Using Discrete Geometry

Measuring and Relieving the Over-Smoothing Problem for Graph Neural Networks from the Topological View

Curvature constrained MPNNs : Improving message passing with local structural properties

Revisiting Over-smoothing in Deep GCNs

Analyzing Neural Network Robustness Using Graph Curvature

Re-Think and Re-Design Graph Neural Networks in Spaces of Continuous Graph Diffusion Functionals