Abstract:Unsupervised/self-supervised pre-training methods for graph representation learning have recently attracted increasing research interests, and they are shown to be able to generalize to various downstream applications. Yet, the adversarial robustness of such pre-trained graph learning models remains largely unexplored. More importantly, most existing defense techniques designed for end-to-end graph representation learning methods require pre-specified label definitions, and thus cannot be directly applied to the pre-training methods. In this paper, we propose an unsupervised defense technique to robustify pre-trained deep graph models, so that the perturbations on the input graph can be successfully identified and blocked before the model is applied to different downstream tasks. Specifically, we introduce a mutual information-based measure, graph representation vulnerability (GRV), to quantify the robustness of graph encoders on the representation space. We then formulate an optimization problem to learn the graph representation by carefully balancing the trade-off between the expressive power and the robustness (i.e., GRV) of the graph encoder. The discrete nature of graph topology and the joint space of graph data make the optimization problem intractable to solve. To handle the above difficulty and to reduce computational expense, we further relax the problem and thus provide an approximate solution. Additionally, we explore a provable connection between the robustness of the unsupervised graph encoder and that of models on downstream tasks. Extensive experiments demonstrate that even without access to labels and tasks, our model is still able to enhance robustness against adversarial attacks on three downstream tasks (node classification, link prediction, and community detection) by an average of +16.5% compared with existing methods.

Self-supervised robust Graph Neural Networks against noisy graphs and noisy labels

DiscoGNN: A Sample-Efficient Framework for Self-Supervised Graph Representation Learning

Unsupervised Adversarially-Robust Representation Learning on Graphs

Robustness of Deep Learning Models on Graphs: A Survey

Robust Training of Graph Neural Networks via Noise Governance

Towards Robust Graph Neural Networks against Label Noise

Towards Robust Graph Neural Networks for Noisy Graphs with Sparse Labels

GNN Cleaner: Label Cleaner for Graph Structured Data

Noise-resistant Graph Neural Networks with Manifold Consistency and Label Consistency

Self-supervised Graph Representations with Generative Adversarial Learning

A Noise-resistant Graph Neural Network by Semi-supervised Contrastive Learning

Robustness of Graph Classification: failure modes, causes, and noise-resistant loss in Graph Neural Networks

Learning on Graphs under Label Noise

Robust graph neural networks with Dirichlet regularization and residual connection

Self-consistent Graph Neural Networks for Semi-supervised Node Classification

Robust Graph Neural Network based on Graph Denoising

You Can't Ignore Either: Unifying Structure and Feature Denoising for Robust Graph Learning

Graph Stochastic Neural Networks for Semi-supervised Learning.

Graph Neural Networks with Coarse- and Fine-Grained Division for Mitigating Label Sparsity and Noise

Dual GNNs: Graph Neural Network Learning with Limited Supervision