Abstract:Surrogate models are often used as alternatives to considerably reduce the computational burden of the expensive computer simulations that are required for engineering designs. The development of surrogate models for complex relationships between the parameters often requires the modeling of high-dimensional functions with limited information, and it is challenging to choose an effective surrogate model over the unknown design space. To this end, the ensemble modelscombined with different surrogate modelsoffer effective solutions. This paper presents a new ensemble model based on the least squares method, which is a regularization strategy and an augmentation strategy; we call the model the regularized least squares ensemble model (RLS-EM). Three individual surrogate modelsKriging, radial basis function, and support vector regressionare used to compose the RLS-EM. Further, the weight factors are estimated by the least squares method without using the global or local error metrics, which are used in most existing methods. To solve the collinearity in the least squares calculation process, a regularization strategy and an augmentation strategy are developed. The two strategies help explore the unknown regions and improve the accuracy on one hand; on the other hand, the collinearity can be reduced, and the overfitting phenomenon that may occur can be avoided. Six numerical functions, from two-dimensional to 12-dimensional, and a computer numerical control (CNC) milling machine bed design problem are used to verify the proposed method. The results of the numerical examples show that RLS-EM saves a considerable amount of computation time while ensuring the same level of robustness and accuracy compared with other ensemble models. The RLS-EM used for the CNC milling machine bed design problem also shows good accuracy characteristics compared with other ensemble methods.

Ensemble Methods for Convex Regression with Applications to Geometric Programming Based Circuit Design

Disciplined Geodesically Convex Programming

Randomized Geometric Algebra Methods for Convex Neural Networks

A Unified Algorithmic Framework for Distributed Composite Optimization.

Convex Relaxation Regression: Black-Box Optimization of Smooth Functions by Learning Their Convex Envelopes

Convex Discrete Optimization

Stochastic Geometric Optimization with Joint Probabilistic Constraints

On Optimizing Ensemble Models using Column Generation

Learning Convex Optimization Models

A Least Squares Ensemble Model Based on Regularization and Augmentation Strategy

Uncertain random geometric programming problems

A Novel Gaussian Process Surrogate Model with Expected Prediction Error for Optimization under Constraints

Improved Convex and Concave Relaxations of Composite Bilinear Forms

Exact Set-valued Estimation using Constrained Convex Generators for uncertain Linear Systems

The Geometry and Well-Posedness of Sparse Regularized Linear Regression

A Smooth Monte Carlo Approach to Joint Chance-Constrained Programs

Numerical Methods for Convex Multistage Stochastic Optimization

Rectangular Chance Constrained Geometric Optimization

Stochastic Methods in Variational Inequalities: Ergodicity, Bias and Refinements

Convex Regression in Multidimensions: Suboptimality of Least Squares Estimators

Ensemble Classification and Regression-Recent Developments, Applications and Future Directions [review Article]