Abstract:This work studies ensemble learning for graph neural networks (GNNs) under the popular semi-supervised setting. Ensemble learning has shown superiority in improving the accuracy and robustness of traditional machine learning by combining the outputs of multiple weak learners. However, adopting a similar idea to integrate different GNN models is challenging because of two reasons. First, GNN is notorious for its poor inference ability, so naively assembling multiple GNN models would deteriorate the inference efficiency. Second, when GNN models are trained with few labeled nodes, their performance are limited. In this case, the vanilla ensemble approach, e.g., majority vote, may be sub-optimal since most base models, i.e., GNNs, may make the wrong predictions. To this end, in this paper, we propose an efficient ensemble learner--E2GNN to assemble multiple GNNs in a learnable way by leveraging both labeled and unlabeled nodes. Specifically, we first pre-train different GNN models on a given data scenario according to the labeled nodes. Next, instead of directly combing their outputs for label inference, we train a simple multi-layer perceptron--MLP model to mimic their predictions on both labeled and unlabeled nodes. Then the unified MLP model is deployed to infer labels for unlabeled or new nodes. Since the predictions of unlabeled nodes from different GNN models may be incorrect, we develop a reinforced discriminator to effectively filter out those wrongly predicted nodes to boost the performance of MLP. By doing this, we suggest a principled approach to tackle the inference issues of GNN ensembles and maintain the merit of ensemble learning: improved performance. Comprehensive experiments over both transductive and inductive settings, across different GNN backbones and 8 benchmark datasets, demonstrate the superiority of E2GNN.

Ensemble of GA Based Selective Neural Network Ensembles

Genetic Algorithm based Selective Neural Network Ensemble.

COMBINING REGRESSION ESTIMATORS: GA-BASED SELECTIVE NEURAL NETWORK ENSEMBLE

Optimizing Weights by Genetic Algorithm for Neural Network Ensemble

Genetic Ensemble of Extreme Learning Machine

Dynamically Weighted Ensemble Neural Networks with Generalized Regression Neural Network for Solving Regression Problems

Selectively ensembling neural classifiers

Ensembling Neural Networks: Many Could Be Better Than All

Dynamically Weighted Ensemble Neural Networks for Regression Problems

The Diversified Ensemble Neural Network.

E2GNN: Efficient Graph Neural Network Ensembles for Semi-Supervised Classification

GNN-Ensemble: Towards Random Decision Graph Neural Networks

Group Ensemble: Learning an Ensemble of ConvNets in a single ConvNet

Research and development of neural network ensembles: a survey

Ensemble Learning for Graph Neural Networks.

Ensemble-Based Distributed Learning for Generative Adversarial Networks.

SAE: Single Architecture Ensemble Neural Networks

Selective Ensemble Of Decision Trees

Neural Subnetwork Ensembles

Neural Network Ensembles: Theory, Training, and the Importance of Explicit Diversity

Graph ensemble neural network