Abstract:Surrogate models are often used as alternatives to considerably reduce the computational burden of the expensive computer simulations that are required for engineering designs. The development of surrogate models for complex relationships between the parameters often requires the modeling of high-dimensional functions with limited information, and it is challenging to choose an effective surrogate model over the unknown design space. To this end, the ensemble modelscombined with different surrogate modelsoffer effective solutions. This paper presents a new ensemble model based on the least squares method, which is a regularization strategy and an augmentation strategy; we call the model the regularized least squares ensemble model (RLS-EM). Three individual surrogate modelsKriging, radial basis function, and support vector regressionare used to compose the RLS-EM. Further, the weight factors are estimated by the least squares method without using the global or local error metrics, which are used in most existing methods. To solve the collinearity in the least squares calculation process, a regularization strategy and an augmentation strategy are developed. The two strategies help explore the unknown regions and improve the accuracy on one hand; on the other hand, the collinearity can be reduced, and the overfitting phenomenon that may occur can be avoided. Six numerical functions, from two-dimensional to 12-dimensional, and a computer numerical control (CNC) milling machine bed design problem are used to verify the proposed method. The results of the numerical examples show that RLS-EM saves a considerable amount of computation time while ensuring the same level of robustness and accuracy compared with other ensemble models. The RLS-EM used for the CNC milling machine bed design problem also shows good accuracy characteristics compared with other ensemble methods.

Combining Neural-Based Regression Predictors Using An Unbiased And Normalized Linear Ensemble Model

Dynamically Weighted Ensemble Neural Networks for Regression Problems

Dynamically Weighted Ensemble Neural Networks with Generalized Regression Neural Network for Solving Regression Problems

Optimizing Weights by Genetic Algorithm for Neural Network Ensemble

Unbiased Linear Neural-Based Fusion with Normalized Weighted Average Algorithm for Regression

A hybrid ensemble method with negative correlation learning for regression

Ranking and Combining Latent Structured Predictive Scores without Labeled Data

Adaptive Linear and Normalized Combination of Radial Basis Function Networks for Function Approximation and Regression

COMBINING REGRESSION ESTIMATORS: GA-BASED SELECTIVE NEURAL NETWORK ENSEMBLE

Adaptively Fusing Neural Network Predictors Toward Higher Accuracy: A Case Study

A Least Squares Ensemble Model Based on Regularization and Augmentation Strategy

A Novel Neural Network Ensemble Method Based on Affinity Propagation Clustering and Lagrange Multiplier

Remaining useful life estimation based on selective ensemble of deep neural networks with diversity

Cocktail Ensemble for Regression

Bagging.LMS: A bagging-based linear fusion with least-mean-square error update for regression

Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning.

Linear Regression Based Combination of Neural Classifiers.

Regularized Negative Correlation Learning for Neural Network Ensembles

Optimal Forecast Combination Based on Neural Networks for Time Series Forecasting

Dynamic ensemble of regression neural networks based on predictive uncertainty

A Double Penalty Model for Ensemble Learning