Abstract:Recent studies reveal the connection between GNNs and the diffusion process, which motivates many diffusion-based GNNs to be proposed. However, since these two mechanisms are closely related, one fundamental question naturally arises: Is there a general diffusion framework that can formally unify these GNNs? The answer to this question can not only deepen our understanding of the learning process of GNNs, but also may open a new door to design a broad new class of GNNs. In this paper, we propose a general diffusion equation framework with the fidelity term, which formally establishes the relationship between the diffusion process with more GNNs. Meanwhile, with this framework, we identify one characteristic of graph diffusion networks, i.e., the current neural diffusion process only corresponds to the first-order diffusion equation. However, by an experimental investigation, we show that the labels of high-order neighbors actually exhibit monophily property, which induces the similarity based on labels among high-order neighbors without requiring the similarity among first-order neighbors. This discovery motives to design a new high-order neighbor-aware diffusion equation, and derive a new type of graph diffusion network (HiD-Net) based on the framework. With the high-order diffusion equation, HiD-Net is more robust against attacks and works on both homophily and heterophily graphs. We not only theoretically analyze the relation between HiD-Net with high-order random walk, but also provide a theoretical convergence guarantee. Extensive experimental results well demonstrate the effectiveness of HiD-Net over state-of-the-art graph diffusion networks.

Nonlinear Sheaf Diffusion in Graph Neural Networks

Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs

Joint Diffusion Processes as an Inductive Bias in Sheaf Neural Networks

Graph Neural Diffusion Networks for Semi-supervised Learning

Diffusion Improves Graph Learning

Flexible Graph Neural Diffusion with Latent Class Representation Learning

Re-Think and Re-Design Graph Neural Networks in Spaces of Continuous Graph Diffusion Functionals

Distributed Diffusion Adaptation Over Graph Signals

Towards understanding Diffusion Models (on Graphs)

Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization

Graph Anisotropic Diffusion

Sparse Training of Discrete Diffusion Models for Graph Generation

From Continuous Dynamics to Graph Neural Networks: Neural Diffusion and Beyond

A Generalized Neural Diffusion Framework on Graphs

Sheaf Hypergraph Networks

Distributed Functional Link Adaptive Filtering for Nonlinear Graph Signal Processing

Graph Neural Aggregation-diffusion with Metastability

A nonlinear diffusion method for semi-supervised learning on hypergraphs

Optimization-Induced Graph Implicit Nonlinear Diffusion

Dissecting the Diffusion Process in Linear Graph Convolutional Networks

Enhanced Expressivity in Graph Neural Networks with Lanczos-Based Linear Constraints