Abstract:Memory replay, which stores a subset of historical data from previous tasks to replay while learning new tasks, exhibits state-of-the-art performance for various continual learning applications on the Euclidean data. While topological information plays a critical role in characterizing graph data, existing memory replay-based graph learning techniques only store individual nodes for replay and do not consider their associated edge information. To this end, based on the message-passing mechanism in graph neural networks (GNNs), we present the Ricci curvature-based graph sparsification technique to perform continual graph representation learning. Specifically, we first develop the subgraph episodic memory (SEM) to store the topological information in the form of computation subgraphs. Next, we sparsify the subgraphs such that they only contain the most informative structures (nodes and edges). The informativeness is evaluated with the Ricci curvature, a theoretically justified metric to estimate the contribution of neighbors to represent a target node. In this way, we can reduce the memory consumption of a computation subgraph from O(dL) to O(1) and enable GNNs to fully utilize the most informative topological information for memory replay. Besides, to ensure the applicability on large graphs, we also provide the theoretically justified surrogate for the Ricci curvature in the sparsification process, which can greatly facilitate the computation. Finally, our empirical studies show that SEM outperforms state-of-the-art approaches significantly on four different public datasets. Unlike existing methods, which mainly focus on task incremental learning (task-IL) setting, SEM also succeeds in the challenging class incremental learning (class-IL) setting in which the model is required to distinguish all learned classes without task indicators and even achieves comparable performance to joint training, which is the performance upper bound for continual learning.

Ising on the Graph: Task-specific Graph Subsampling via the Ising Model

Interpretable Sparsification of Brain Graphs: Better Practices and Effective Designs for Graph Neural Networks

Sensor selection on graphs via data-driven node sub-sampling in network time series

Graph sub-sampling for divide-and-conquer algorithms in large networks

Learning Graphs from Linear Measurements: Fundamental Trade-offs and Applications

Sampling and Recovery of Graph Signals based on Graph Neural Networks

Subsampling of Correlated Graph Signals

Graph Sampling for Scalable and Expressive Graph Neural Networks on Homophilic Graphs

Edge Sampling of Graphs: Graph Signal Processing Approach With Edge Smoothness

Dynamic Sampling from Graphical Models.

Tensorizing Subgraph Search in the Supernet

Sampling Subgraph Network with Application to Graph Classification

Graph-Structured Sparse Optimization for Connected Subgraph Detection

Graph Coarsening with Neural Networks

Sparsified Subgraph Memory for Continual Graph Representation Learning

Graph Sparsifications using Neural Network Assisted Monte Carlo Tree Search

Sampling and Reconstruction of Signals on Product Graphs

Understanding Negative Sampling in Graph Representation Learning

GraphSAINT: Graph Sampling Based Inductive Learning Method

Ricci Curvature-Based Graph Sparsification for Continual Graph Representation Learning.

Improving the interpretability of GNN predictions through conformal-based graph sparsification