Abstract:Graph contrastive learning (GCL) alleviates the heavy reliance on label information for graph representation learning (GRL) via self-supervised learning schemes. The core idea is to learn by maximising mutual information for similar instances, which requires similarity computation between two node instances. However, GCL is inefficient in both time and memory consumption. In addition, GCL normally requires a large number of training epochs to be well-trained on large-scale datasets. Inspired by an observation of a technical defect (i.e., inappropriate usage of Sigmoid function) commonly used in two representative GCL works, DGI and MVGRL, we revisit GCL and introduce a new learning paradigm for self-supervised graph representation learning, namely, Group Discrimination (GD), and propose a novel GD-based method called Graph Group Discrimination (GGD). Instead of similarity computation, GGD directly discriminates two groups of node samples with a very simple binary cross-entropy loss. In addition, GGD requires much fewer training epochs to obtain competitive performance compared with GCL methods on large-scale datasets. These two advantages endow GGD with very efficient property. Extensive experiments show that GGD outperforms state-of-the-art self-supervised methods on eight datasets. In particular, GGD can be trained in 0.18 seconds (6.44 seconds including data preprocessing) on ogbn-arxiv, which is orders of magnitude (10,000+) faster than GCL baselines while consuming much less memory. Trained with 9 hours on ogbn-papers100M with billion edges, GGD outperforms its GCL counterparts in both accuracy and efficiency.

What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper aims to address the efficiency issues of Graph Contrastive Learning (GCL) in Graph Representation Learning (GRL). Specifically, GCL methods face the following problems when dealing with large-scale datasets: 1. **High Time and Memory Consumption**: GCL methods require a significant amount of time and memory resources during training, especially when handling large-scale datasets. 2. **Many Training Epochs**: GCL methods typically need a large number of training epochs to achieve good performance on large-scale datasets. 3. **Technical Deficiencies**: Some representative GCL methods (such as DGI and MVGRL) have technical deficiencies in their implementation, such as the improper use of the Sigmoid function, leading to actual performance that does not meet expectations. To address these issues, the authors re-examined existing GCL methods and introduced a new self-supervised graph representation learning paradigm—Group Discrimination (GD). By directly discriminating between groups of positive and negative samples instead of computing similarity, the GD method can significantly improve training efficiency, reduce memory consumption, and perform well on large-scale datasets. ### Specific Improvements 1. **Group Discrimination Paradigm**: The GD method learns graph representations by directly discriminating between groups of positive and negative samples, rather than maximizing mutual information between similar instances as in GCL. This approach avoids complex similarity calculations, thereby improving training efficiency. 2. **Fast Convergence**: The GD method requires only a few training epochs to achieve competitive performance on large-scale datasets. 3. **Efficiency**: Experimental results show that the GD method is several orders of magnitude faster in training time compared to existing GCL methods across multiple datasets, while also consuming much less memory. ### Experimental Validation The authors conducted extensive experiments on multiple benchmark datasets, including small-scale, medium-scale, and large-scale datasets. The experimental results demonstrate that the model based on GD (Graph Group Discrimination, GGD) achieves state-of-the-art performance while significantly outperforming existing GCL methods in terms of training time and memory consumption. Notably, GGD performs exceptionally well when handling large-scale datasets (such as ogbn-papers100M), with training times thousands of times faster than the most efficient GCL baseline methods. ### Conclusion By introducing the Group Discrimination paradigm, the paper addresses the efficiency issues of existing GCL methods when dealing with large-scale datasets, providing a faster, more efficient, and scalable self-supervised graph representation learning method.

Rethinking and Scaling Up Graph Contrastive Learning: An Extremely Efficient Approach with Group Discrimination

Debiased Graph Contrastive Learning.

SimGRACE: A Simple Framework for Graph Contrastive Learning Without Data Augmentation

Structure-Aware Group Discrimination with Adaptive-View Graph Encoder: A Fast Graph Contrastive Learning Framework

Rethinking Graph Contrastive Learning: an Efficient Single-View Approach Via Instance Discrimination

AdaGCL: Adaptive Subgraph Contrastive Learning to Generalize Large-scale Graph Training

MDGCL: Graph Contrastive Learning Framework with Multiple Graph Diffusion Methods

Towards Equivariant Graph Contrastive Learning via Cross-Graph Augmentation

FastGCL: Fast Self-Supervised Learning on Graphs via Contrastive Neighborhood Aggregation

Graph Contrastive Learning with Node-Level Accurate Difference

Graph Soft-Contrastive Learning via Neighborhood Ranking

Rethinking and Simplifying Bootstrapped Graph Latents

Unifying Graph Contrastive Learning with Flexible Contextual Scopes

DPGCL: Dual pass filtering based graph contrastive learning

Augmentation-Free Graph Contrastive Learning of Invariant-Discriminative Representations

CSGCL: Community-Strength-Enhanced Graph Contrastive Learning

Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck

GA-GGD: Improving semantic discriminability in graph contrastive learning via Generative Adversarial Network

Graph Contrastive Representation Learning with Input-Aware and Cluster-Aware Regularization

Contrastive Learning Meets Homophily: Two Birds with One Stone.

Relation-aware Graph Contrastive Learning