Abstract:Curriculum learning provides a systematic approach to training. It refines training progressively, tailors training to task requirements, and improves generalization through exposure to diverse examples. We present a curriculum learning approach that builds on existing knowledge about text and graph complexity formalisms for training with text graph data. The core part of our approach is a novel data scheduler, which employs "spaced repetition" and complexity formalisms to guide the training process. We demonstrate the effectiveness of the proposed approach on several text graph tasks and graph neural network architectures. The proposed model gains more and uses less data; consistently prefers text over graph complexity indices throughout training, while the best curricula derived from text and graph complexity indices are equally effective; and it learns transferable curricula across GNN models and datasets. In addition, we find that both node-level (local) and graph-level (global) graph complexity indices, as well as shallow and traditional text complexity indices play a crucial role in effective curriculum learning.

What problem does this paper attempt to address?

The paper primarily aims to address the challenges faced when training Graph Neural Networks (GNNs) on text graph data, particularly how to effectively organize the training process to improve model performance. Specifically, the paper proposes a complexity-guided curriculum learning framework (TGCL) to tackle the problem through the following aspects: 1. **Design of Curriculum Learning Method**: The paper introduces a new curriculum learning framework that leverages existing formal knowledge about text and graph complexity to guide the training of GNNs on text graph data. The core contribution is a novel data scheduler that combines "spaced repetition" techniques with formal complexity knowledge to progressively guide the training process. 2. **Efficient Training of GNNs**: By quantifying sample difficulty and gradually introducing training samples to GNNs based on these complexity metrics, TGCL can train more efficiently while enhancing the model's generalization ability. 3. **Dynamic Scheduling Strategy**: The paper develops a dynamic scheduling strategy, which involves learning delay parameters for each complexity metric to decide when to reintroduce these metrics into the training process. This strategy can dynamically adjust based on the model's learning difficulty for specific sample sets. 4. **Experimental Validation**: The authors extensively validate the effectiveness of the proposed TGCL method through experiments. The experiments cover real-world text graph datasets and various GNN architectures, achieving significant performance improvements in link prediction and node classification tasks. In summary, the goal of the paper is to design a new curriculum learning framework to enhance the efficiency and effectiveness of training GNNs on text graph data. This approach not only considers the complexity characteristics of the data itself but also the dynamic changes in model learning, thereby achieving better training resource allocation and model performance.

Complexity-Guided Curriculum Learning for Text Graphs

Curriculum Learning for Graph Neural Networks: A Multiview Competence-based Approach

Multi-Modal Curriculum Learning over Graphs

Curriculum Graph Machine Learning: A Survey

Curriculum Learning for Graph Neural Networks: Which Edges Should We Learn First

Enhancing Temporal Knowledge Graph Representation with Curriculum Learning

Multi-Task Curriculum Graph Contrastive Learning with Clustering Entropy Guidance

Ling-CL: Understanding NLP Models through Linguistic Curricula

An efficient curriculum learning-based strategy for molecular graph learning

Enhancing Signed Graph Neural Networks through Curriculum-Based Training

Analyzing Curriculum Learning for Sentiment Analysis along Task Difficulty, Pacing and Visualization Axes

Visualizing and Understanding Curriculum Learning for Long Short-Term Memory Networks

Curriculum Learning: A Survey

A Comprehensive Survey on Curriculum Learning

Coarse-to-Fine Curriculum Learning

An Analytical Theory of Curriculum Learning in Teacher-Student Networks

Curriculum Learning for Small Code Language Models

Curriculum Learning with a Progression Function

Learning to Learn: How to Continuously Teach Humans and Machines

Easy Begun is Half Done: Spatial-Temporal Graph Modeling with ST-Curriculum Dropout

When Do Curricula Work?