Abstract:Network representation learning (NRL) has far-reaching effects on data mining research, showing its importance in many real-world applications. NRL, also known as network embedding, aims at preserving graph structures in a low-dimensional space. These learned representations can be used for subsequent machine learning tasks, such as vertex classification, link prediction, and data visualization. Recently, graph convolutional network (GCN)-based models, e.g., GraphSAGE, have drawn a lot of attention for their success in inductive NRL. When conducting unsupervised learning on large-scale graphs, some of these models employ negative sampling (NS) for optimization, which encourages a target vertex to be close to its neighbors while being far from its negative samples. However, NS draws negative vertices through a random pattern or based on the degrees of vertices. Thus, the generated samples could be either highly relevant or completely unrelated to the target vertex. Moreover, as the training goes, the gradient of NS objective calculated with the inner product of the unrelated negative samples and the target vertex may become zero, which will lead to learning inferior representations. To address these problems, we propose an adversarial training method tailored for unsupervised inductive NRL on large networks. For efficiently keeping track of high-quality negative samples, we design a caching scheme with sampling and updating strategies that has a wide exploration of vertex proximity while considering training costs. Besides, the proposed method is adaptive to various existing GCN-based models without significantly complicating their optimization process. Extensive experiments show that our proposed method can achieve better performance compared with the state-of-the-art models.

AdaNS: Adaptive Negative Sampling for Unsupervised Graph Representation Learning

Understanding Negative Sampling in Graph Representation Learning

Self-Supervised Node Representation Learning Via Node-to-Neighbourhood Alignment.

Negative Sampling Strategy Based on Multi-Hop Neighbors for Graph Representation Learning

Negative samples selecting strategy for graph contrastive learning

HNS: Hierarchical Negative Sampling for Network Representation Learning

AFANS: Augmentation-Free Graph Contrastive Learning with Adversarial Negative Sampling

Layer-diverse Negative Sampling for Graph Neural Networks

Towards a better negative sampling strategy for dynamic graphs

Boosting Graph Contrastive Learning Via Adaptive Sampling

Leveraging Network Structure for Efficient Dynamic Negative Sampling in Network Embedding

Structure Aware Negative Sampling in Knowledge Graphs

Adversarial Caching Training: Unsupervised Inductive Network Representation Learning on Large-Scale Graphs

RW-NSGCN: A Robust Approach to Structural Attacks via Negative Sampling

Diffusion-based Negative Sampling on Graphs for Link Prediction

Select The Best: Enhancing Graph Representation with Adaptive Negative Sample Selection

ENSG: Enhancing Negative Sampling in Graph Convolutional Networks for Recommendation Systems

Graph Alignment with Noisy Supervision

Self-supervised Graph Representations with Generative Adversarial Learning

A Negative Sample-Free Graph Contrastive Learning Algorithm

Adversarial Training Regularization for Negative Sampling Based Network Embedding