Neighbor Contrastive Learning on Learnable Graph Augmentation

Xiao Shen,Dewang Sun,Shirui Pan,Xi Zhou,Laurence T. Yang
2023-06-02
Abstract:Recent years, graph contrastive learning (GCL), which aims to learn representations from unlabeled graphs, has made great progress. However, the existing GCL methods mostly adopt human-designed graph augmentations, which are sensitive to various graph datasets. In addition, the contrastive losses originally developed in computer vision have been directly applied to graph data, where the neighboring nodes are regarded as negatives and consequently pushed far apart from the anchor. However, this is contradictory with the homophily assumption of networks that connected nodes often belong to the same class and should be close to each other. In this work, we propose an end-to-end automatic GCL method, named NCLA to apply neighbor contrastive learning on learnable graph augmentation. Several graph augmented views with adaptive topology are automatically learned by the multi-head graph attention mechanism, which can be compatible with various graph datasets without prior domain knowledge. In addition, a neighbor contrastive loss is devised to allow multiple positives per anchor by taking network topology as the supervised signals. Both augmentations and embeddings are learned end-to-end in the proposed NCLA. Extensive experiments on the benchmark datasets demonstrate that NCLA yields the state-of-the-art node classification performance on self-supervised GCL and even exceeds the supervised ones, when the labels are extremely limited. Our code is released at <a class="link-external link-https" href="https://github.com/shenxiaocam/NCLA" rel="external noopener nofollow">this https URL</a>.
Social and Information Networks
What problem does this paper attempt to address?
The paper aims to address two main issues in Graph Contrastive Learning (GCL): 1. **Lack of generalization in manually designed graph augmentation methods**: Most existing GCL methods adopt manually designed graph augmentation strategies, which are highly sensitive to different graph datasets and require the selection of appropriate augmentation methods based on prior knowledge or trial and error. This limits the efficiency and generality of the methods. 2. **Inappropriateness of directly applying contrastive loss from computer vision**: Existing GCL methods directly apply contrastive loss from the field of computer vision to graph data, ignoring the topological structure characteristics of graph data. In graph data, neighboring nodes often belong to the same category and should be similar to each other rather than distant. However, the contrastive loss in existing methods treats neighboring nodes as negative samples to be pushed apart, which contradicts the homogeneity assumption of Graph Neural Networks (GNNs). To address the above issues, the authors propose a new end-to-end automatic GCL method—NCLA (Neighbor Contrastive Learning on Learnable Graph Augmentation). This method automatically generates graph augmentation views with adaptive topological structures through a multi-head graph attention mechanism and designs a new neighbor contrastive loss function that allows each anchor point to have multiple positive samples, thereby better utilizing the topological information of the graph. Experimental results show that NCLA achieves significantly better performance than existing methods in semi-supervised node classification tasks on multiple benchmark datasets.