Abstract:Memory replay, which stores a subset of historical data from previous tasks to replay while learning new tasks, exhibits state-of-the-art performance for various continual learning applications on the Euclidean data. While topological information plays a critical role in characterizing graph data, existing memory replay-based graph learning techniques only store individual nodes for replay and do not consider their associated edge information. To this end, based on the message-passing mechanism in graph neural networks (GNNs), we present the Ricci curvature-based graph sparsification technique to perform continual graph representation learning. Specifically, we first develop the subgraph episodic memory (SEM) to store the topological information in the form of computation subgraphs. Next, we sparsify the subgraphs such that they only contain the most informative structures (nodes and edges). The informativeness is evaluated with the Ricci curvature, a theoretically justified metric to estimate the contribution of neighbors to represent a target node. In this way, we can reduce the memory consumption of a computation subgraph from O(dL) to O(1) and enable GNNs to fully utilize the most informative topological information for memory replay. Besides, to ensure the applicability on large graphs, we also provide the theoretically justified surrogate for the Ricci curvature in the sparsification process, which can greatly facilitate the computation. Finally, our empirical studies show that SEM outperforms state-of-the-art approaches significantly on four different public datasets. Unlike existing methods, which mainly focus on task incremental learning (task-IL) setting, SEM also succeeds in the challenging class incremental learning (class-IL) setting in which the model is required to distinguish all learned classes without task indicators and even achieves comparable performance to joint training, which is the performance upper bound for continual learning.

Sparsified Subgraph Memory for Continual Graph Representation Learning

Ricci Curvature-Based Graph Sparsification for Continual Graph Representation Learning.

Sufficient Subgraph Embedding Memory for Continual Graph Representation Learning

Topology-aware Embedding Memory for Learning on Expanding Graphs.

Graph Continual Learning with Debiased Lossless Memory Replay

Topology-aware Embedding Memory for Continual Learning on Expanding Networks

Graph Memory Learning: Imitating Lifelong Remembering and Forgetting of Brain Networks

Holistic Memory Diversification for Incremental Learning in Growing Graphs

SpanGNN: Towards Memory-Efficient Graph Neural Networks via Spanning Subgraph Training

Contrastive Augmented Graph2Graph Memory Interaction for Few Shot Continual Learning

Dual Memory Model for Experience-Once Task-Incremental Lifelong Learning.

Summarizing Stream Data for Memory-Constrained Online Continual Learning

The Effectiveness of Memory Replay in Large Scale Continual Learning

M EMORY R EPLAY WITH D ATA C OMPRESSION FOR C ONTINUAL L EARNING

Schematic Memory Persistence and Transience for Efficient and Robust Continual Learning

Enhancing Graph Neural Networks via Memorized Global Information

Saliency-Guided Hidden Associative Replay for Continual Learning

FTF-ER: Feature-Topology Fusion-Based Experience Replay Method for Continual Graph Learning

E-CGL: An Efficient Continual Graph Learner

Memory Recall: A Simple Neural Network Training Framework Against Catastrophic Forgetting

Triple-Memory Networks: A Brain-Inspired Method for Continual Learning