Abstract:Unsupervised/self-supervised pre-training methods for graph representation learning have recently attracted increasing research interests, and they are shown to be able to generalize to various downstream applications. Yet, the adversarial robustness of such pre-trained graph learning models remains largely unexplored. More importantly, most existing defense techniques designed for end-to-end graph representation learning methods require pre-specified label definitions, and thus cannot be directly applied to the pre-training methods. In this paper, we propose an unsupervised defense technique to robustify pre-trained deep graph models, so that the perturbations on the input graph can be successfully identified and blocked before the model is applied to different downstream tasks. Specifically, we introduce a mutual information-based measure, graph representation vulnerability (GRV), to quantify the robustness of graph encoders on the representation space. We then formulate an optimization problem to learn the graph representation by carefully balancing the trade-off between the expressive power and the robustness (i.e., GRV) of the graph encoder. The discrete nature of graph topology and the joint space of graph data make the optimization problem intractable to solve. To handle the above difficulty and to reduce computational expense, we further relax the problem and thus provide an approximate solution. Additionally, we explore a provable connection between the robustness of the unsupervised graph encoder and that of models on downstream tasks. Extensive experiments demonstrate that even without access to labels and tasks, our model is still able to enhance robustness against adversarial attacks on three downstream tasks (node classification, link prediction, and community detection) by an average of +16.5% compared with existing methods.

Disentangled Text Representation Learning With Information-Theoretic Perspective for Adversarial Robustness

Unsupervised Adversarially-Robust Representation Learning on Graphs

Improving Robustness and Generality of NLP Models Using Disentangled Representations

Disentangling Factors of Variation in Deep Representations Using Adversarial Training.

Exploring Robust Features for Improving Adversarial Robustness

Learning Adversarially Robust Representations via Worst-Case Mutual Information Maximization

Improving Adversarial Robustness via Mutual Information Estimation

Robustness Exploration of Semantic Information in Adversarial Training

Semantically Consistent Visual Representation for Adversarial Robustness

Unsupervised Adversarial Perturbation Eliminating Via Disentangled Representations.

Improving Model Robustness Against Adversarial Examples with Redundant Fully Connected Layer.

Improving the Adversarial Robustness of NLP Models by Information Bottleneck

Improving adversarial robustness of deep neural networks by using semantic information

Towards Adversarial Robustness with Multidimensional Perturbations Via Contrastive Learning

Disentangled Representation Learning with Transmitted Information Bottleneck

Dynamically Disentangling Social Bias from Task-Oriented Representations with Adversarial Attack.

Toward Enhanced Robustness in Unsupervised Graph Representation Learning: A Graph Information Bottleneck Perspective

Singular Regularization with Information Bottleneck Improves Model's Adversarial Robustness

Robust Textual Embedding Against Word-level Adversarial Attacks

Disentangled Contrastive Learning for Learning Robust Textual Representations

AFD: Mitigating Feature Gap for Adversarial Robustness by Feature Disentanglement