Abstract:Memory replay, which stores a subset of historical data from previous tasks to replay while learning new tasks, exhibits state-of-the-art performance for various continual learning applications on the Euclidean data. While topological information plays a critical role in characterizing graph data, existing memory replay-based graph learning techniques only store individual nodes for replay and do not consider their associated edge information. To this end, based on the message-passing mechanism in graph neural networks (GNNs), we present the Ricci curvature-based graph sparsification technique to perform continual graph representation learning. Specifically, we first develop the subgraph episodic memory (SEM) to store the topological information in the form of computation subgraphs. Next, we sparsify the subgraphs such that they only contain the most informative structures (nodes and edges). The informativeness is evaluated with the Ricci curvature, a theoretically justified metric to estimate the contribution of neighbors to represent a target node. In this way, we can reduce the memory consumption of a computation subgraph from O(dL) to O(1) and enable GNNs to fully utilize the most informative topological information for memory replay. Besides, to ensure the applicability on large graphs, we also provide the theoretically justified surrogate for the Ricci curvature in the sparsification process, which can greatly facilitate the computation. Finally, our empirical studies show that SEM outperforms state-of-the-art approaches significantly on four different public datasets. Unlike existing methods, which mainly focus on task incremental learning (task-IL) setting, SEM also succeeds in the challenging class incremental learning (class-IL) setting in which the model is required to distinguish all learned classes without task indicators and even achieves comparable performance to joint training, which is the performance upper bound for continual learning.

Subgraph Reconstruction via Reversible Subgraph Embedding.

RESTORE: Graph Embedding Assessment Through Reconstruction

Knowledge Graph Embedding with Diversity of Structures

Refined Graph Encoder Embedding via Self-Training and Latent Community Recovery

Detecting Small Query Graphs in A Large Graph via Neural Subgraph Search

A Dilated Recurrent Neural Network-Based Model for Graph Embedding

Graph Game Embedding

Enhanced Sub-graph Reconstruction Graph Neural Network for Recommendation

Self-supervised Reconstructed Graph Learning for Link Prediction in Bipartite Graphs

Reconstructive Sequence-Graph Network for Video Summarization

Supervised Hypergraph Reconstruction

Learning node embeddings via summary graphs: a brief theoretical analysis

Ricci Curvature-Based Graph Sparsification for Continual Graph Representation Learning.

Community-Centric Graph Unlearning

Scalable Graph Embedding for Asymmetric Proximity.

Reconstruction of multiplex networks via graph embeddings

Collaborative bi-aggregation for directed graph embedding

Sparsified Subgraph Memory for Continual Graph Representation Learning

Deep Graph Neural Networks via Flexible Subgraph Aggregation

Near-optimal Steiner Tree Computation Powered by Node Embeddings

RelaGraph: Improving Embedding on Small-Scale Sparse Knowledge Graphs by Neighborhood Relations