Graph Contrastive Learning with Generative Adversarial Network

Cheng Wu,Chaokun Wang,Jingcao Xu,Ziyang Liu,Kai Zheng,Xiaowei Wang,Yang Song,Kun Gai

DOI: https://doi.org/10.1145/3580305.3599370

2023-08-01

Abstract:Graph Neural Networks (GNNs) have demonstrated promising results on exploiting node representations for many downstream tasks through supervised end-to-end training. To deal with the widespread label scarcity issue in real-world applications, Graph Contrastive Learning (GCL) is leveraged to train GNNs with limited or even no labels by maximizing the mutual information between nodes in its augmented views generated from the original graph. However, the distribution of graphs remains unconsidered in view generation, resulting in the ignorance of unseen edges in most existing literature, which is empirically shown to be able to improve GCL's performance in our experiments. To this end, we propose to incorporate graph generative adversarial networks (GANs) to learn the distribution of views for GCL, in order to i) automatically capture the characteristic of graphs for augmentations, and ii) jointly train the graph GAN model and the GCL model. Specifically, we present GACN, a novel Generative Adversarial Contrastive learning Network for graph representation learning. GACN develops a view generator and a view discriminator to generate augmented views automatically in an adversarial style. Then, GACN leverages these views to train a GNN encoder with two carefully designed self-supervised learning losses, including the graph contrastive loss and the Bayesian personalized ranking Loss. Furthermore, we design an optimization framework to train all GACN modules jointly. Extensive experiments on seven real-world datasets show that GACN is able to generate high-quality augmented views for GCL and is superior to twelve state-of-the-art baseline methods. Noticeably, our proposed GACN surprisingly discovers that the generated views in data augmentation finally conform to the well-known preferential attachment rule in online networks.

Machine Learning

What problem does this paper attempt to address?

The paper attempts to address the problem of how to utilize Generative Adversarial Networks (GAN) to enhance model performance in Graph Contrastive Learning (GCL). Specifically, due to the scarcity of labeled data in real-world applications, GCL trains Graph Neural Networks (GNN) by maximizing the mutual information between different augmented views of the original graph, without or with only a small amount of labels. However, existing GCL methods ignore the distribution characteristics of the graph when generating views, leading to a failure to fully exploit potential new edges to improve model performance.

To solve the above problem, the authors propose a method that combines Graph Generative Adversarial Networks (Graph GAN) with Graph Contrastive Learning, namely GACN (Generative Adversarial Contrastive learning Network). GACN achieves this through the following ways:
1. **Automatically capturing graph features**: A Generative Adversarial Network is designed to automatically learn the distribution of the graph and generate high-quality augmented views.
2. **Joint training**: An optimization framework is proposed that can jointly train the Graph GAN model and the GCL model, so that the generated views can both deceive the GAN discriminator and be effectively encoded by the GCL model.

Experimental results show that GACN outperforms existing methods on multiple real-world datasets, and the generated views conform to the well-known preferential attachment rule.