Abstract:Graph Neural Networks (GNNs) have shown superior performance in node classification. However, GNNs perform poorly in the Few-Shot Node Classification (FSNC) task that requires robust generalization to make accurate predictions for unseen classes with limited labels. To tackle the challenge, we propose the integration of Sharpness-Aware Minimization (SAM)--a technique designed to enhance model generalization by finding a flat minimum of the loss landscape--into GNN training. The standard SAM approach, however, consists of two forward-backward steps in each training iteration, doubling the computational cost compared to the base optimizer (e.g., Adam). To mitigate this drawback, we introduce a novel algorithm, Fast Graph Sharpness-Aware Minimization (FGSAM), that integrates the rapid training of Multi-Layer Perceptrons (MLPs) with the superior performance of GNNs. Specifically, we utilize GNNs for parameter perturbation while employing MLPs to minimize the perturbed loss so that we can find a flat minimum with good generalization more efficiently. Moreover, our method reutilizes the gradient from the perturbation phase to incorporate graph topology into the minimization process at almost zero additional cost. To further enhance training efficiency, we develop FGSAM+ that executes exact perturbations periodically. Extensive experiments demonstrate that our proposed algorithm outperforms the standard SAM with lower computational costs in FSNC tasks. In particular, our FGSAM+ as a SAM variant offers a faster optimization than the base optimizer in most cases. In addition to FSNC, our proposed methods also demonstrate competitive performance in the standard node classification task for heterophilic graphs, highlighting the broad applicability. The code is available at <a class="link-external link-https" href="https://github.com/draym28/FGSAM_NeurIPS24" rel="external noopener nofollow">this https URL</a>.

Graph augmentation for node-level few-shot learning

MuL-GRN: Multi-Level Graph Relation Network for Few-Shot Node Classification

NodeAug: Semi-Supervised Node Classification with Data Augmentation

Graph Few-shot Learning via Knowledge Transfer

Fast Graph Sharpness-Aware Minimization for Enhancing and Accelerating Few-Shot Node Classification

Graph Contrastive Learning Meets Graph Meta Learning: A Unified Method for Few-shot Node Tasks

Node Classification on Graphs with Few-Shot Novel Labels Via Meta Transformed Network Embedding

Graph-based few-shot learning with transformed feature propagation and optimal class allocation

Graph Data Augmentation for Node Classification

InfoMax Classification-Enhanced Learnable Network for Few-Shot Node Classification

Graph Prototypical Networks for Few-shot Learning on Attributed Networks

Adaptive-Step Graph Meta-Learner for Few-Shot Graph Classification

Meta-GPS++: Enhancing Graph Meta-Learning with Contrastive Learning and Self-Training

Fuzzy Graph Neural Network for Few-Shot Learning

CPT: Competence-progressive Training Strategy for Few-shot Node Classification

Two-level Graph Network for Few-Shot Class-Incremental Learning

Hybrid Graph Neural Networks for Few-Shot Learning

Structure-Aware Prototypical Neural Process for Few-Shot Graph Classification

GraphSR: A Data Augmentation Algorithm for Imbalanced Node Classification

Exploring Graph Classification Techniques Under Low Data Constraints: A Comprehensive Study

Topological enhanced graph neural networks for semi-supervised node classification