Unifying Graph Contrastive Learning via Graph Message Augmentation

Ziyan Zhang,Bo Jiang,Jin Tang,Bin Luo
2024-01-08
Abstract:Graph contrastive learning is usually performed by first conducting Graph Data Augmentation (GDA) and then employing a contrastive learning pipeline to train GNNs. As we know that GDA is an important issue for graph contrastive learning. Various GDAs have been developed recently which mainly involve dropping or perturbing edges, nodes, node attributes and edge attributes. However, to our knowledge, it still lacks a universal and effective augmentor that is suitable for different types of graph data. To address this issue, in this paper, we first introduce the graph message representation of graph data. Based on it, we then propose a novel Graph Message Augmentation (GMA), a universal scheme for reformulating many existing GDAs. The proposed unified GMA not only gives a new perspective to understand many existing GDAs but also provides a universal and more effective graph data augmentation for graph self-supervised learning tasks. Moreover, GMA introduces an easy way to implement the mixup augmentor which is natural for images but usually challengeable for graphs. Based on the proposed GMA, we then propose a unified graph contrastive learning, termed Graph Message Contrastive Learning (GMCL), that employs attribution-guided universal GMA for graph contrastive learning. Experiments on many graph learning tasks demonstrate the effectiveness and benefits of the proposed GMA and GMCL approaches.
Machine Learning,Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
### Problems the paper attempts to solve This paper aims to solve an important problem in Graph Contrastive Learning (GCL): the lack of a general and effective Graph Data Augmentation (GDA) method. Specifically: 1. **Limitations of existing GDA methods**: - Current GDA methods usually need to be customized for different graph data types and lack a general method applicable to all graph data types. - When dealing with graph data, existing GDA methods usually introduce noise or lose label - invariant information, which will affect the performance of GCL. 2. **The proposed new method**: - The author introduced Graph Message Representation and, on this basis, proposed Graph Message Augmentation (GMA). - GMA can not only unify multiple existing GDA methods but also provide a more effective and general graph data augmentation scheme. - GMA also provides a natural way to implement the mixup operation on graphs, which is a commonly used technique in image data augmentation but more difficult for graph data. 3. **Objectives**: - Through GMA, the author hopes to provide a general and effective graph data augmentation method that can be applied to various graph learning tasks and improve the performance of graph contrastive learning. ### Main contributions 1. **Proposing Graph Message Augmentation (GMA)**: - GMA provides a new graph data augmentation method by performing dropout, perturbation, and mixup operations on graph messages instead of directly on the original graph. - GMA can unify multiple existing GDA methods and provide a unified understanding framework. 2. **Mixup enhancer based on graph message representation**: - A simple and effective graph message mixup method for graph data augmentation and graph contrastive learning tasks is proposed. 3. **Graph Message Contrastive Learning (GMCL) architecture**: - Based on GMA, a simple and unified contrastive learning architecture GMCL for self - supervised learning of graph data is proposed. 4. **Learnable Attribute - Guided Graph Message Augmentor (AttGMA)**: - A new learnable module AttGMA is designed to adaptively learn graph message augmentation metrics to preserve label - invariant information in the augmented data. ### Experimental results The experimental results show that the proposed GMCL method outperforms traditional GDA methods on multiple graph datasets, verifying its effectiveness and superiority. ### Summary This paper solves the problem of the lack of a general and effective graph data augmentation method in graph contrastive learning by introducing graph message representation and graph message augmentation, providing a new and more effective solution for graph learning tasks.