Abstract:In the realm of graph out-of-distribution(OOD), despite recent strides in advancing graph neural networks (GNNs) for the modeling of graph data, training GNNs on large-scale datasets presents a formidable hurdle due to the pervasive challenge of overfitting. To address these issues, researchers have explored adversarial training, a technique that enriches training data with worst-case adversarial examples. However, while prior work on adversarial training primarily focuses on safeguarding GNNs against malicious attacks, its potential to enhance the OOD generalization abilities of GNNs in the context of graph analytics remains less explored. In our research, we delve into the inner workings of GNNs by examining the landscapes of weight and feature losses, which respectively illustrate how the loss function changes concerning model weights and node features. Our investigation reveals a noteworthy phenomenon: GNNs are inclined to become trapped in sharp local minima within these loss landscapes, resulting in suboptimal OOD generalization performance. To address this challenge, we introduce the concept of co-adversarial perturbation optimization, which considers both model weights and node features, and we design an alternating adversarial perturbation algorithm for graph out-of-distribution generalization. This algorithm operates iteratively, smoothing the weight and feature loss landscapes alternately. Moreover, our training process unfolds in two distinct stages. The first stage centers on standard cross-entropy minimization, ensuring rapid convergence of GNN models. In the second stage, we employ our alternating adversarial training strategy to prevent the models from becoming ensnared in locally sharp minima. Our extensive experiments provide compelling evidence that our CAP approach can generally enhance the OOD generalization performance of GNNs across a diverse range of large-scale graphs.

Adversarial Training for Graph Neural Networks via Graph Subspace Energy Optimization

Unsupervised Adversarially-Robust Representation Learning on Graphs

Spectral Adversarial Training for Robust Graph Neural Network

Graph Adversarial Training: Dynamically Regularizing Based on Graph Structure.

Robust Graph Convolutional Networks with Directional Graph Adversarial Training.

Adversarial Detection on Graph Structured Data.

HC-Ref: Hierarchical Constrained Refinement for Robust Adversarial Training of GNNs

A Two-Stage Co-Adversarial Perturbation to Mitigate Out-of-distribution Generalization of Large-Scale Graph

GARNET: Reduced-Rank Topology Learning for Robust and Scalable Graph Neural Networks

Smoothing Adversarial Training for GNN

A Dual Robust Graph Neural Network Against Graph Adversarial Attacks

CAP: Co-Adversarial Perturbation on Weights and Features for Improving Generalization of Graph Neural Networks

Adversarial Defense Framework for Graph Neural Network

Adversarial Attack on Large Scale Graph

Defending Against Adversarial Attacks on Graph Neural Networks Via Similarity Property.

Deep Insights into Graph Adversarial Learning: An Empirical Study Perspective.

Adversarial Training Methods for Network Embedding.

Adversarial Training on Weights for Graph Neural Networks.

ERGCN: Data Enhancement-Based Robust Graph Convolutional Network Against Adversarial Attacks.

Bayesian Adversarial Attack on Graph Neural Networks (student Abstract)

Adversarial Graph Augmentation to Improve Graph Contrastive Learning