Abstract:Subgraph recognition aims at discovering a compressed substructure of a graph that is most informative to the graph property. It can be formulated by optimizing Graph Information Bottleneck (GIB) with a mutual information estimator. However, GIB suffers from training instability and degenerated results due to its intrinsic optimization process. To tackle these issues, we reformulate the subgraph recognition problem into two steps: graph perturbation and subgraph selection, leading to a novel Variational Graph Information Bottleneck (VGIB) framework. VGIB first employs the noise injection to modulate the information flow from the input graph to the perturbed graph. Then, the perturbed graph is encouraged to be informative to the graph property. VGIB further obtains the desired subgraph by filtering out the noise in the perturbed graph. With the customized noise prior for each input, the VGIB objective is endowed with a tractable variational upper bound, leading to a superior empirical performance as well as theoretical properties. Extensive experiments on graph interpretation, explainability of Graph Neural Networks, and graph classification show that VGIB finds better subgraphs than existing methods. Code is avaliable at <a class="link-external link-https" href="https://github.com/Samyu0304/VGIB" rel="external noopener nofollow">this https URL</a>

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to improve sub - graph recognition to discover the most informative compressed sub - structures for graph properties. Specifically, the existing Graph Information Bottleneck (GIB) methods have problems of training instability and result degradation during the optimization process, which are mainly caused by their inherent mutual information estimation process. To solve these problems, the authors propose a new framework - Variational Graph Information Bottleneck (VGIB), which overcomes the shortcomings of GIB methods by reformulating the sub - graph recognition problem into two steps: graph perturbation and sub - graph selection. ### Main problems and solutions 1. **Existing problems**: - The GIB method leads to training instability and unsatisfactory results during the optimization process because it needs to estimate Mutual Information. - The inaccurate estimation of mutual information further affects the effect of sub - graph recognition. 2. **Solutions**: - **Introducing the VGIB framework**: Improve sub - graph recognition through two steps of graph perturbation and sub - graph selection. - **Graph perturbation**: Inject noise into the input graph to regulate the information flow and generate a perturbed graph. - **Sub - graph selection**: Filter out the noise from the perturbed graph and extract the required sub - graph. - **Customizing noise prior**: Design a specific Gaussian noise prior for each input graph, making the VGIB objective function have a tractable variational upper bound. - **Optimization method**: Use the Gumbel - Softmax reparameterization technique to handle discrete noise injection operations and optimize by the gradient descent method. ### Specific implementation - **Formula representation**: - Mutual information is defined as: \[ I(X, Y)=\int_X\int_Y p(x, y)\log\frac{p(x, y)}{p(x)p(y)}dxdy \] - GIB objective function: \[ \min_{G_{\text{sub}}} - I(G_{\text{sub}}, Y)+\beta I(G_{\text{sub}}, G) \] - VGIB objective function: \[ \min_{G_N}-I(G_N, Y)+\beta I(G_N, G) \] - Loss function: \[ L = L_{\text{cls}}(G_N, Y)+\beta L_{\text{MI}}(Z_N, G) \] ### Experimental verification The authors verified the effectiveness of VGIB through multiple experiments, including graph interpretation, the interpretability of graph neural networks, and graph classification tasks. The experimental results show that VGIB is superior to existing methods in both optimization efficiency and sub - graph recognition effect. ### Conclusion By introducing the VGIB framework, the authors successfully solved the problems of training instability and result degradation in the GIB method and significantly improved the effect of sub - graph recognition. This improvement not only improves the performance of the model but also provides new ideas for the interpretability research of graph neural networks.

Improving Subgraph Recognition with Variational Graph Information Bottleneck

Graph Structure Learning with Variational Information Bottleneck

Utilizing Edge Features in Graph Neural Networks Via Variational Information Maximization

Discovering the Representation Bottleneck of Graph Neural Networks from Multi-order Interactions

Towards Consistency and Complementarity: A Multiview Graph Information Bottleneck Approach

CCGIB: A Cross-Channel Graph Information Bottleneck Principle

Dynamic Graph Information Bottleneck

Individual and Structural Graph Information Bottlenecks for Out-of-Distribution Generalization

Towards Reliable Link Prediction with Robust Graph Information Bottleneck

Uncertainty in the Variational Information Bottleneck

Invariant Graph Learning Meets Information Bottleneck for Out-of-Distribution Generalization

Robust Node Representation Learning via Graph Variational Diffusion Networks

V-InFoR: A Robust Graph Neural Networks Explainer for Structurally Corrupted Graphs.

Graph Information Vanishing Phenomenon in Implicit Graph Neural Networks

Incorporating Retrieval-based Causal Learning with Information Bottlenecks for Interpretable Graph Neural Networks

Discovering the Representation Bottleneck of Graph Neural Networks

Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck

Hierarchical Graph Neural Network Based-on Semi-implicit Variational Inference

Conditional Graph Information Bottleneck for Molecular Relational Learning

Graph Information Bottleneck for Remote Sensing Segmentation

Towards Explanation for Unsupervised Graph-Level Representation Learning