Abstract:Memory replay, which stores a subset of historical data from previous tasks to replay while learning new tasks, exhibits state-of-the-art performance for various continual learning applications on the Euclidean data. While topological information plays a critical role in characterizing graph data, existing memory replay-based graph learning techniques only store individual nodes for replay and do not consider their associated edge information. To this end, based on the message-passing mechanism in graph neural networks (GNNs), we present the Ricci curvature-based graph sparsification technique to perform continual graph representation learning. Specifically, we first develop the subgraph episodic memory (SEM) to store the topological information in the form of computation subgraphs. Next, we sparsify the subgraphs such that they only contain the most informative structures (nodes and edges). The informativeness is evaluated with the Ricci curvature, a theoretically justified metric to estimate the contribution of neighbors to represent a target node. In this way, we can reduce the memory consumption of a computation subgraph from O(dL) to O(1) and enable GNNs to fully utilize the most informative topological information for memory replay. Besides, to ensure the applicability on large graphs, we also provide the theoretically justified surrogate for the Ricci curvature in the sparsification process, which can greatly facilitate the computation. Finally, our empirical studies show that SEM outperforms state-of-the-art approaches significantly on four different public datasets. Unlike existing methods, which mainly focus on task incremental learning (task-IL) setting, SEM also succeeds in the challenging class incremental learning (class-IL) setting in which the model is required to distinguish all learned classes without task indicators and even achieves comparable performance to joint training, which is the performance upper bound for continual learning.

Robust Graph Representation Learning Via Neural Sparsification.

Unsupervised Adversarially-Robust Representation Learning on Graphs

Heterogeneous Graph Sparsification for Efficient Representation Learning

Graph Sparsification for Enhanced Conformal Prediction in Graph Neural Networks

Graph Neural Network Meets Sparse Representation: Graph Sparse Neural Networks Via Exclusive Group Lasso

Interpretable Sparsification of Brain Graphs: Better Practices and Effective Designs for Graph Neural Networks

Two Heads Are Better Than One: Boosting Graph Sparse Training via Semantic and Topological Awareness

Graph Sparsifications using Neural Network Assisted Monte Carlo Tree Search

Graph Sparsification via Mixture of Graphs

Robust Graph Representation Learning for Local Corruption Recovery

Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization

Training Sparse Graph Neural Networks Via Pruning and Sprouting

Robust graph neural networks with Dirichlet regularization and residual connection

Robust Network Learning Via Inverse Scale Variational Sparsification

Ricci Curvature-Based Graph Sparsification for Continual Graph Representation Learning.

Scalable Graph Neural Networks for Heterogeneous Graphs

GARNET: Reduced-Rank Topology Learning for Robust and Scalable Graph Neural Networks

Graph Representation Learning on Noise and Sparse Labels.

Feature Expansion for Graph Neural Networks

Improving the interpretability of GNN predictions through conformal-based graph sparsification

Robust Graph Structure Learning with the Alignment of Features and Adjacency Matrix