Abstract:Unsupervised/self-supervised pre-training methods for graph representation learning have recently attracted increasing research interests, and they are shown to be able to generalize to various downstream applications. Yet, the adversarial robustness of such pre-trained graph learning models remains largely unexplored. More importantly, most existing defense techniques designed for end-to-end graph representation learning methods require pre-specified label definitions, and thus cannot be directly applied to the pre-training methods. In this paper, we propose an unsupervised defense technique to robustify pre-trained deep graph models, so that the perturbations on the input graph can be successfully identified and blocked before the model is applied to different downstream tasks. Specifically, we introduce a mutual information-based measure, graph representation vulnerability (GRV), to quantify the robustness of graph encoders on the representation space. We then formulate an optimization problem to learn the graph representation by carefully balancing the trade-off between the expressive power and the robustness (i.e., GRV) of the graph encoder. The discrete nature of graph topology and the joint space of graph data make the optimization problem intractable to solve. To handle the above difficulty and to reduce computational expense, we further relax the problem and thus provide an approximate solution. Additionally, we explore a provable connection between the robustness of the unsupervised graph encoder and that of models on downstream tasks. Extensive experiments demonstrate that even without access to labels and tasks, our model is still able to enhance robustness against adversarial attacks on three downstream tasks (node classification, link prediction, and community detection) by an average of +16.5% compared with existing methods.

Improving the Robustness of Wasserstein Embedding by Adversarial PAC-Bayesian Learning

Unsupervised Adversarially-Robust Representation Learning on Graphs

Wasserstein Adversarially Regularized Graph Autoencoder

Identity-Preserving Adversarial Training for Robust Network Embedding

Wasserstein distributional robustness of neural networks

Understanding and Improvement of Adversarial Training for Network Embedding from an Optimization Perspective

Adversarial Robustness of Probabilistic Network Embedding for Link Prediction

A Unified Wasserstein Distributional Robustness Framework for Adversarial Training

Deep Variational Network Embedding in Wasserstein Space

Regularization for Adversarial Robust Learning

Adversarially Regularized Graph Autoencoder for Graph Embedding

A Robust and Generalized Framework for Adversarial Graph Embedding

Robust Graph Learning Under Wasserstein Uncertainty

Adversarial Robustness under Long-Tailed Distribution

Stacked Wasserstein Autoencoder

A PAC-Bayes Analysis of Adversarial Robustness

Robustness-Aware Word Embedding Improves Certified Robustness to Adversarial Word Substitutions

Improving Adversarial Robustness via Promoting Ensemble Diversity.

A View-Adversarial Framework for Multi-View Network Embedding

Improving Adversarial Robustness Via Promoting Ensemble Diversity

Advances in Knowledge Discovery and Data Mining - 23rd Pacific-Asia Conference, PAKDD 2019, Macau, China, April 14-17, 2019, Proceedings, Part I.