Abstract:In a data-decentralized environment where data are distributed across individual local nodes, inter-node data sharing is often restricted due to security concerns and communication costs. Under these restrictions, using entire data to train a global predictive model is challenging. A viable strategy is to leverage local predictive models trained independently on local nodes. However, this adversely affect generalization performance, particularly when the data distribution varies between local nodes. To address this issue in regression problems, we propose a dynamic ensemble method that combines the predictions of locally trained neural networks without the need to access data from local nodes. For a query instance, the prediction and its uncertainty from each neural network are computed by applying Monte Carlo dropout. Subsequently, smaller/larger weights are assigned to neural networks with higher/lower predictive uncertainty. The final prediction result is derived using the weighted average of the individual predictions. The proposed method is applicable to a data-decentralized environment where data are inaccessible and thus cannot be shared across local nodes. Transmitting query instances to local nodes and receiving predictions and weights from the local nodes are the only requirements, thereby ensuring low communication costs. Experimental results on simulated data-decentralized environments using benchmark regression datasets validated the superior generalization performance of the proposed method compared with baseline methods. The proposed method was found to be more effective for regression neural networks that demonstrate higher performance at their respective local nodes.

Dynamically Weighted Ensemble Neural Networks with Generalized Regression Neural Network for Solving Regression Problems

Dynamically Weighted Ensemble Neural Networks for Regression Problems

Optimizing Weights by Genetic Algorithm for Neural Network Ensemble

Support Vector Machines Ensemble With Optimizing Weights By Genetic Algorithm

COMBINING REGRESSION ESTIMATORS: GA-BASED SELECTIVE NEURAL NETWORK ENSEMBLE

The Diversified Ensemble Neural Network.

Locally Weighted Ensemble Learning for Regression.

Ensembling Neural Networks: Many Could Be Better Than All

Improving Deep Neural Network Ensembles Using Reconstruction Error

Optimizing Ensemble Weights and Hyperparameters of Machine Learning Models for Regression Problems

An Online Learning Neural Network Ensembles with Random Weights for Regression of Sequential Data Stream

Robust Modeling of Unknown Dynamical Systems via Ensemble Averaged Learning

Genetic Algorithm based Selective Neural Network Ensemble.

Dynamic ensemble of regression neural networks based on predictive uncertainty

A hybrid ensemble method with negative correlation learning for regression

Ensemble of GA Based Selective Neural Network Ensembles

UA-DRN: Unbiased Aggregation of Deep Neural Networks for Regression Ensemble

End-to-End Ensemble Learning by Exploiting the Correlation Between Individuals and Weights.

A Novel Neural Network Ensemble Method Based on Affinity Propagation Clustering and Lagrange Multiplier

Combining Neural-Based Regression Predictors Using An Unbiased And Normalized Linear Ensemble Model

Boost Neural Networks by Checkpoints