Abstract:Masked graph autoencoder (MGAE) has emerged as a promising self-supervised graph pre-training (SGP) paradigm due to its simplicity and effectiveness. However, existing efforts perform the mask-then-reconstruct operation in the raw data space as is done in computer vision (CV) and natural language processing (NLP) areas, while neglecting the important non-Euclidean property of graph data. As a result, the highly unstable local connection structures largely increase the uncertainty in inferring masked data and decrease the reliability of the exploited self-supervision signals, leading to inferior representations for downstream evaluations. To address this issue, we propose a novel SGP method termed Robust mAsked gRaph autoEncoder (RARE) to improve the certainty in inferring masked data and the reliability of the self-supervision mechanism by further masking and reconstructing node samples in the high-order latent feature space. Through both theoretical and empirical analyses, we have discovered that performing a joint mask-then-reconstruct strategy in both latent feature and raw data spaces could yield improved stability and performance. To this end, we elaborately design a masked latent feature completion scheme, which predicts latent features of masked nodes under the guidance of high-order sample correlations that are hard to be observed from the raw data perspective. Specifically, we first adopt a latent feature predictor to predict the masked latent features from the visible ones. Next, we encode the raw data of masked samples with a momentum graph encoder and subsequently employ the resulting representations to improve predicted results through latent feature matching. Extensive experiments on seventeen datasets have demonstrated the effectiveness and robustness of RARE against state-of-the-art (SOTA) competitors across three downstream tasks.

Masked Graph Auto-Encoder Constrained Graph Pooling

Graph explicit pooling for graph-level representation learning

Careful Selection and Thoughtful Discarding: Graph Explicit Pooling Utilizing Discarded Nodes

MGPOOL: multi-granular graph pooling convolutional networks representation learning

GSAPool: Gated Structure Aware Pooling for Graph Representation Learning.

Hierarchical Graph Pooling with Self-Adaptive Cluster Aggregation

On Exploring Node-feature and Graph-structure Diversities for Node Drop Graph Pooling

Diversified Node Sampling based Hierarchical Transformer Pooling for Graph Representation Learning

Distribution Knowledge Embedding for Graph Pooling

ReiPool: Reinforced Pooling Graph Neural Networks for Graph-Level Representation Learning

RARE: Robust Masked Graph Autoencoder

Geometric Pooling: Maintaining More Representative Information

DMSPool: Dual Multi-Scale Pooling for Graph Representation Learning

Geometric Pooling: maintaining more useful information

Graph Pooling for Graph Neural Networks: Progress, Challenges, and Opportunities

WGDPool: A broad scope extraction for weighted graph data

DIPool: Degree-Induced Pooling for Hierarchical Graph Representation Learning.

Pooling in Graph Convolutional Neural Networks

Seeing All From a Few: Nodes Selection Using Graph Pooling for Graph Clustering

MxPool: Multiplex Pooling for Hierarchical Graph Representation Learning

Not All Edges Are Peers: Accurate Structure-Aware Graph Pooling Networks