Abstract:Continual learning has emerged as a crucial paradigm for learning from sequential data while preserving previous knowledge. In the realm of continual graph learning, where graphs continuously evolve based on streaming graph data, continual graph learning presents unique challenges that require adaptive and efficient graph learning methods in addition to the problem of catastrophic forgetting. The first challenge arises from the interdependencies between different graph data, where previous graphs can influence new data distributions. The second challenge lies in the efficiency concern when dealing with large graphs. To addresses these two problems, we produce an Efficient Continual Graph Learner (E-CGL) in this paper. We tackle the interdependencies issue by demonstrating the effectiveness of replay strategies and introducing a combined sampling strategy that considers both node importance and diversity. To overcome the limitation of efficiency, E-CGL leverages a simple yet effective MLP model that shares weights with a GCN during training, achieving acceleration by circumventing the computationally expensive message passing process. Our method comprehensively surpasses nine baselines on four graph continual learning datasets under two settings, meanwhile E-CGL largely reduces the catastrophic forgetting problem down to an average of -1.1%. Additionally, E-CGL achieves an average of 15.83x training time acceleration and 4.89x inference time acceleration across the four datasets. These results indicate that E-CGL not only effectively manages the correlation between different graph data during continual training but also enhances the efficiency of continual learning on large graphs. The code is publicly available at <a class="link-external link-https" href="https://github.com/aubreygjh/E-CGL" rel="external noopener nofollow">this https URL</a>.

Ensemble Gaussian Processes for Online Learning Over Graphs With Adaptivity and Scalability

Online Graph Learning under Smoothness Priors

Weighted Ensembles for Adaptive Active Learning

Online Learning Of Expanding Graphs

Online Graph Learning via Time-Vertex Adaptive Filters: From Theory to Cardiac Fibrillation

Whom to Trust? Elective Learning for Distributed Gaussian Process Regression

Spatio-temporal learning with the online finite and infinite echo-state Gaussian processes

Graph Neural Networks Gone Hogwild

Stochastic Deep Gaussian Processes over Graphs

Enhancing Graph Self-Supervised Learning with Graph Interplay

Online Proximal ADMM for Graph Learning from Streaming Smooth Signals

Ensemble Kalman Filtering for Online Gaussian Process Regression and Learning

Online scalable Gaussian processes with conformal prediction for guaranteed coverage

Online Graph Filtering Over Expanding Graphs

E-CGL: An Efficient Continual Graph Learner

Gaussian Process with Graph Convolutional Kernel for Relational Learning

Efficient And Robust Semi-Supervised Learning Over A Sparse-Regularized Graph

Efficient Learning Algorithms for Gaussian Processes

Decentralized Eigendecomposition for Online Learning over Graphs with Applications

Adaptive Sparse Gaussian Process

Sparsity-Aware Distributed Learning for Gaussian Processes with Linear Multiple Kernel