Abstract:Textual graphs are ubiquitous in real-world applications, featuring rich text information with complex relationships, which enables advanced research across various fields. Textual graph representation learning aims to generate low-dimensional feature embeddings from textual graphs that can improve the performance of downstream tasks. A high-quality feature embedding should effectively capture both the structural and the textual information in a textual graph. However, most textual graph dataset benchmarks rely on word2vec techniques to generate feature embeddings, which inherently limits their capabilities. Recent works on textual graph representation learning can be categorized into two folds: supervised and unsupervised methods. Supervised methods finetune a language model on labeled nodes, which have limited capabilities when labeled data is scarce. Unsupervised methods, on the other hand, extract feature embeddings by developing complex training pipelines. To address these limitations, we propose a novel unified unsupervised learning autoencoder framework, named Node Level Graph AutoEncoder (NodeGAE). We employ language models as the backbone of the autoencoder, with pretraining on text reconstruction. Additionally, we add an auxiliary loss term to make the feature embeddings aware of the local graph structure. Our method maintains simplicity in the training process and demonstrates generalizability across diverse textual graphs and downstream tasks. We evaluate our method on two core graph representation learning downstream tasks: node classification and link prediction. Comprehensive experiments demonstrate that our approach substantially enhances the performance of diverse graph neural networks (GNNs) across multiple textual graph datasets.

Link Prediction and Node Classification Based on Multitask Graph Autoencoder.

Self-Supervised Node Representation Learning Via Node-to-Neighbourhood Alignment.

Graph autoencoder for directed weighted network

Effective Model Integration Algorithm for Improving Link and Sign Prediction in Complex Networks

Stabilizing and Enhancing Link Prediction through Deepened Graph Auto-Encoders

Deep Dynamic Network Embedding for Link Prediction

Improved Graph Autoencoder for Network Reconstruction

Deep Feature Learning of Multi-Network Topology for Node Classification

Dynamic network link prediction with node representation learning from graph convolutional networks

Continuous-time Dynamic Network Node Classification Algorithm Based on Autoencoder

Link Prediction via Graph Attention Network

A Weighted Symmetric Graph Embedding Approach for Link Prediction in Undirected Graphs

A Complex Network based Graph Embedding Method for Link Prediction

Node Level Graph Autoencoder: Unified Pretraining for Textual Graph Learning

Link prediction in heterogeneous information networks: An improved deep graph convolution approach

Transferable graph auto-encoders for cross-network node classification

Dynamic link prediction by learning the representation of node-pair via graph neural networks

Multi-view network embedding with node similarity ensemble

A Bayesian Graph Embedding Model for Link-Based Classification Problems

Link Prediction Via Subgraph Embedding-Based Convex Matrix Completion

Heterogeneous Hypergraph Variational Autoencoder for Link Prediction