Abstract:Memory replay, which stores a subset of historical data from previous tasks to replay while learning new tasks, exhibits state-of-the-art performance for various continual learning applications on the Euclidean data. While topological information plays a critical role in characterizing graph data, existing memory replay-based graph learning techniques only store individual nodes for replay and do not consider their associated edge information. To this end, based on the message-passing mechanism in graph neural networks (GNNs), we present the Ricci curvature-based graph sparsification technique to perform continual graph representation learning. Specifically, we first develop the subgraph episodic memory (SEM) to store the topological information in the form of computation subgraphs. Next, we sparsify the subgraphs such that they only contain the most informative structures (nodes and edges). The informativeness is evaluated with the Ricci curvature, a theoretically justified metric to estimate the contribution of neighbors to represent a target node. In this way, we can reduce the memory consumption of a computation subgraph from O(dL) to O(1) and enable GNNs to fully utilize the most informative topological information for memory replay. Besides, to ensure the applicability on large graphs, we also provide the theoretically justified surrogate for the Ricci curvature in the sparsification process, which can greatly facilitate the computation. Finally, our empirical studies show that SEM outperforms state-of-the-art approaches significantly on four different public datasets. Unlike existing methods, which mainly focus on task incremental learning (task-IL) setting, SEM also succeeds in the challenging class incremental learning (class-IL) setting in which the model is required to distinguish all learned classes without task indicators and even achieves comparable performance to joint training, which is the performance upper bound for continual learning.

Topology-aware Embedding Memory for Learning on Expanding Graphs.

Topology-aware Embedding Memory for Continual Learning on Expanding Networks

Sufficient Subgraph Embedding Memory for Continual Graph Representation Learning

Sparsified Subgraph Memory for Continual Graph Representation Learning

Ricci Curvature-Based Graph Sparsification for Continual Graph Representation Learning.

Dual Memory Model for Experience-Once Task-Incremental Lifelong Learning.

Graph Continual Learning with Debiased Lossless Memory Replay

FTF-ER: Feature-Topology Fusion-Based Experience Replay Method for Continual Graph Learning

Graph Memory Learning: Imitating Lifelong Remembering and Forgetting of Brain Networks

SpanGNN: Towards Memory-Efficient Graph Neural Networks via Spanning Subgraph Training

Triple-Memory Networks: A Brain-Inspired Method for Continual Learning

Holistic Memory Diversification for Incremental Learning in Growing Graphs

Scalable and Effective Temporal Graph Representation Learning With Hyperbolic Geometry

Enhancing Graph Neural Networks via Memorized Global Information

Memory-Enhanced Period-Aware Graph Neural Network for General POI Recommendation.

Overcoming Catastrophic Forgetting in Graph Neural Networks with Experience Replay

Hierarchical Prototype Networks for Continual Graph Representation Learning

Learning Temporal Interaction Graph Embedding Via Coupled Memory Networks.

M EMORY R EPLAY WITH D ATA C OMPRESSION FOR C ONTINUAL L EARNING

TEAL: New Selection Strategy for Small Buffers in Experience Replay Class Incremental Learning

FreshGNN: Reducing Memory Access via Stable Historical Embeddings for Graph Neural Network Training