Abstract:Recent advancements in graph neural networks (GNNs) have highlighted the critical need of calibrating model predictions, with neighborhood prediction similarity recognized as a pivotal component. Existing studies suggest that nodes with analogous neighborhood prediction similarity often exhibit similar calibration characteristics. Building on this insight, recent approaches incorporate neighborhood similarity into node-wise temperature scaling techniques. However, our analysis reveals that this assumption does not hold universally. Calibration errors can differ significantly even among nodes with comparable neighborhood similarity, depending on their confidence levels. This necessitates a re-evaluation of existing GNN calibration methods, as a single, unified approach may lead to sub-optimal calibration. In response, we introduce **Simi-Mailbox**, a novel approach that categorizes nodes by both neighborhood similarity and their own confidence, irrespective of proximity or connectivity. Our method allows fine-grained calibration by employing *group-specific* temperature scaling, with each temperature tailored to address the specific miscalibration level of affiliated nodes, rather than adhering to a uniform trend based on neighborhood similarity. Extensive experiments demonstrate the effectiveness of our **Simi-Mailbox** across diverse datasets on different GNN architectures, achieving up to 13.79\% error reduction compared to uncalibrated GNN predictions.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the problems in the prediction uncertainty calibration of graph neural networks (GNNs). Specifically, the author points out the limitations of existing GNN calibration methods, especially some problems in node calibration based on neighborhood similarity. #### Main problems: 1. **Limitations of existing calibration methods**: - Existing research assumes that nodes with similar neighborhood prediction similarities usually exhibit similar calibration characteristics, and on this basis, a node temperature scaling technique based on neighborhood similarity is introduced. - However, through analysis, the author finds that even for nodes with similar neighborhood similarities, their calibration errors can change significantly due to different confidence levels. This means that neighborhood similarity alone cannot effectively solve the calibration problem. 2. **Over - confidence and under - confidence**: - Among nodes with similar neighborhood similarities, both over - confident and under - confident situations may occur. This phenomenon has not been fully captured by previous studies, resulting in existing methods being not accurate enough in calibration. 3. **Need for more fine - grained calibration methods**: - Since a single, unified calibration method may lead to sub - optimal results, a method that can perform more fine - grained calibration according to the specific situation of nodes (such as neighborhood similarity and confidence) is needed. #### Solution: To solve the above problems, the author proposes a new post - processing calibration method - **SIMI - MAILBOX**. This method classifies nodes into different groups by considering neighborhood similarity and node confidence simultaneously, and assigns specific temperature parameters to each group for calibration. This method can better handle the calibration differences between different nodes, thereby improving the overall reliability of the model. #### Method overview: 1. **Node classification**: Classify nodes according to neighborhood similarity and confidence to ensure that nodes in the same group have similar calibration errors. 2. **Intra - group temperature scaling**: Assign a specific temperature parameter to each group to adjust the predicted values of nodes in this group, making their confidence more consistent with the true probability. 3. **Optimize temperature parameters**: Optimize these temperature parameters by minimizing the difference between the average confidence and the accuracy rate within each group. Through this series of improvements, SIMI - MAILBOX can achieve higher calibration accuracy on multiple datasets and different GNN architectures, and can reduce the error by up to 13.79% compared with uncalibrated GNN predictions. ### Summary The main contribution of this paper is to reveal the limitations of existing GNN calibration methods and propose a new calibration method - SIMI - MAILBOX, which significantly improves the reliability of GNN predictions by performing fine - grained calibration in combination with neighborhood similarity and confidence.

Towards Precise Prediction Uncertainty in GNNs: Refining GNNs with Topology-grouping Strategy

What Makes Graph Neural Networks Miscalibrated?

SimCalib: Graph Neural Network Calibration based on Similarity between Nodes

On the Temperature of Bayesian Graph Neural Networks for Conformal Prediction

Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration

GETS: Ensemble Temperature Scaling for Calibration in Graph Neural Networks

Valid Conformal Prediction for Dynamic GNNs

Accurate and Scalable Estimation of Epistemic Uncertainty for Graph Neural Networks

On Calibration of Graph Neural Networks for Node Classification

Improving Node Classification Accuracy of GNN through Input and Output Intervention

Similarity-Navigated Conformal Prediction for Graph Neural Networks

Conformalized Link Prediction on Graph Neural Networks

Beyond Probability Partitions: Calibrating Neural Networks with Semantic Aware Grouping

Clarify Confused Nodes via Separated Learning

Improving Uncertainty Calibration of Deep Neural Networks via Truth Discovery and Geometric Optimization

Conditional Uncertainty Quantification for Tensorized Topological Neural Networks

Improving the interpretability of GNN predictions through conformal-based graph sparsification

Uncertainty in GNN Learning Evaluations: The Importance of a Consistent Benchmark for Community Detection

Towards Calibrated Deep Clustering Network

RoCP-GNN: Robust Conformal Prediction for Graph Neural Networks in Node-Classification

Model-Free Local Recalibration of Neural Networks