Abstract:Unsupervised/self-supervised pre-training methods for graph representation learning have recently attracted increasing research interests, and they are shown to be able to generalize to various downstream applications. Yet, the adversarial robustness of such pre-trained graph learning models remains largely unexplored. More importantly, most existing defense techniques designed for end-to-end graph representation learning methods require pre-specified label definitions, and thus cannot be directly applied to the pre-training methods. In this paper, we propose an unsupervised defense technique to robustify pre-trained deep graph models, so that the perturbations on the input graph can be successfully identified and blocked before the model is applied to different downstream tasks. Specifically, we introduce a mutual information-based measure, graph representation vulnerability (GRV), to quantify the robustness of graph encoders on the representation space. We then formulate an optimization problem to learn the graph representation by carefully balancing the trade-off between the expressive power and the robustness (i.e., GRV) of the graph encoder. The discrete nature of graph topology and the joint space of graph data make the optimization problem intractable to solve. To handle the above difficulty and to reduce computational expense, we further relax the problem and thus provide an approximate solution. Additionally, we explore a provable connection between the robustness of the unsupervised graph encoder and that of models on downstream tasks. Extensive experiments demonstrate that even without access to labels and tasks, our model is still able to enhance robustness against adversarial attacks on three downstream tasks (node classification, link prediction, and community detection) by an average of +16.5% compared with existing methods.

Unleashing the Power of Language Models in Text-Attributed Graph

Unleashing the Power of Language Models in Text-Attributed Graph.

Unsupervised Adversarially-Robust Representation Learning on Graphs

Self-Supervised Node Representation Learning Via Node-to-Neighbourhood Alignment.

NGAT: Attention in Breadth and Depth Exploration for Semi-Supervised Graph Representation Learning

UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs

GraphFormers: GNN-nested Language Models for Linked Text Representation

Node Level Graph Autoencoder: Unified Pretraining for Textual Graph Learning

GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph

Graph Transformer Networks Based Text Representation.

Learning Multiplex Representations on Text-Attributed Graphs with One Language Model Encoder

High-Frequency-aware Hierarchical Contrastive Selective Coding for Representation Learning on Text-attributed Graphs

Language Models are Graph Learners

A Pure Transformer Pretraining Framework on Text-attributed Graphs

Pretraining Language Models with Text-Attributed Heterogeneous Graphs

Large Language Models as Topological Structure Enhancers for Text-Attributed Graphs

Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning

Unleashing the Power of Emojis in Texts via Self-supervised Graph Pre-Training

Efficient and effective training of language and graph neural network models

Verbalized Graph Representation Learning: A Fully Interpretable Graph Model Based on Large Language Models Throughout the Entire Process

Learning Effective Road Network Representation with Hierarchical Graph Neural Networks