Abstract:Graph contrastive learning (GCL) aims to learn representations from unlabeled graph data in a self-supervised manner and has developed rapidly in recent years. However, edgelevel contrasts are not well explored by most existing GCL methods. Most studies in GCL only regard edges as auxiliary information while updating node features. One of the primary obstacles of edge-based GCL is the heavy computation burden. To tackle this issue, we propose a model that can efficiently learn edge features for GCL, namely AugmentationFree Edge Contrastive Learning (AFECL) to achieve edgeedge contrast. AFECL depends on no augmentation consisting of two parts. Firstly, we design a novel edge feature generation method, where edge features are computed by embedding concatenation of their connected nodes. Secondly, an edge contrastive learning scheme is developed, where edges connecting the same nodes are defined as positive pairs, and other edges are defined as negative pairs. Experimental results show that compared with recent state-of-the-art GCL methods or even some supervised GNNs, AFECL achieves SOTA performance on link prediction and semi-supervised node classification of extremely scarce labels. The source code is available at <a class="link-external link-https" href="https://github.com/YujunLi361/AFECL" rel="external noopener nofollow">this https URL</a>.
What problem does this paper attempt to address?
### What problems does this paper attempt to solve?
This paper aims to solve the problem that edge - level contrast in Graph Contrastive Learning (GCL) has not been fully explored. Specifically, most of the existing GCL methods only regard edges as auxiliary information to update node features, without making full use of edge information for contrastive learning. In addition, existing methods have an excessive computational burden when dealing with large - scale graph data and rely on data augmentation strategies, which may damage the topological structure of the graph and thus affect the embedding quality.
To solve these problems, the authors propose a model named **Augmentation - Free Edge Contrastive Learning (AFECL)**. The main contributions and goals of this model are as follows:
1. **Study edge - level contrast for the first time**: A new edge representation learning method is introduced. Edge embeddings are generated through node embeddings, and a new edge - level contrast loss function is designed. Among them, edges connecting the same nodes are regarded as positive sample pairs, and edges not connecting nodes are regarded as negative sample pairs.
2. **A new efficient and flexible GCL paradigm**: AFECL does not require additional manual graph augmentation strategies and can be trained on graphs of any scale. Only a small number of edge features need to be generated, thus improving flexibility and efficiency.
3. **Excellent performance**: Experimental results show that compared with the latest GCL methods and even some supervised Graph Neural Networks (GNN), AFECL performs well in link prediction and semi - supervised node classification tasks with very few labels, reaching the state - of - the - art performance (SOTA).
### Key innovation points of the model
- **No need for data augmentation**: AFECL directly uses the original graph as input, avoiding the destruction of the graph structure caused by data augmentation.
- **Edge feature generation**: Edge embeddings are generated by splicing node embeddings, which simplifies the computational complexity.
- **Edge contrast loss**: Positive and negative sample pairs based on edge connection relationships are defined to ensure the effectiveness of contrastive learning.
### Presentation of formulas
To understand the working principle of the model more clearly, the following are the key formulas:
1. **Edge coefficient calculation**:
\[
\alpha^{(k)}_{ij} = \frac{\exp(\text{LeakyReLU}(a^{(k)}[W^{(k)}x_i \| W^{(k)}x_j]))}{\sum_{v_p \in N_i \cup \{v_i\}} \exp(\text{LeakyReLU}(a^{(k)}[W^{(k)}x_i \| W^{(k)}x_p]))}
\]
where \(W^{(k)}\) is the learnable weight matrix of the \(k\)-th head, \(a^{(k)}\) is the learnable weight vector of the \(k\)-th head, and \(\|\) represents the splicing operation.
2. **Node embedding generation**:
\[
h^{(k)}_i = \text{ELU}\left(\sum_{v_j \in N_i \cup \{v_i\}} \alpha^{(k)}_{ij} W^{(k)} x_j\right)
\]
\[
h_i = \parallel_{k = 1}^K h^{(k)}_i
\]
3. **Edge embedding generation**:
\[
h_{ij} = g(e_{ij}) = \varphi(f(v_i), f(v_j)) = \varphi(h_i, h_j)
\]
\[
\varphi(f(v_i), f(v_j)) = W [f(v_i) \| f(v_j)] = \varphi(h_i \| h_j)
\]
4. **Edge contrast loss**:
\[
\ell(h_{ij}) = -\log \left( \frac{\exp(\theta(h_{ij}, h_{ij}) / \tau) + \sum_{v_