Abstract:To relieve the computational cost of design evaluations using expensive finite element simulations, surrogate models have been widely applied in computer-aided engineering design. Machine learning algorithms (MLAs) have been implemented as surrogate models due to their capability of learning the complex interrelations between the design variables and the response from big datasets. Typically, an MLA regression model contains model parameters and hyperparameters. The model parameters are obtained by fitting the training data. Hyperparameters, which govern the model structures and the training processes, are assigned by users before training. There is a lack of systematic studies on the effect of hyperparameters on the accuracy and robustness of the surrogate model. In this work, we proposed to establish a hyperparameter optimization (HOpt) framework to deepen our understanding of the effect. Four frequently used MLAs, namely Gaussian Process Regression (GPR), Support Vector Machine (SVM), Random Forest Regression (RFR), and Artificial Neural Network (ANN), are tested on four benchmark examples. For each MLA model, the model accuracy and robustness before and after the HOpt are compared. The results show that HOpt can generally improve the performance of the MLA models in general. HOpt leads to few improvements in the MLAs accuracy and robustness for complex problems, which are featured by high-dimensional mixed-variable design space. The HOpt is recommended for the design problems with intermediate complexity. We also investigated the additional computational costs incurred by HOpt. The training cost is closely related to the MLA architecture. After HOpt, the training cost of ANN and RFR is increased more than that of the GPR and SVM. To sum up, this study benefits the selection of HOpt method for the different types of design problems based on their complexity.

Parameter Screening Using Impact Factors and Surrogate-Based ANOVA Techniques

Novel Non-Uniform Adaptive Grid Refinement Control Parameterization Approach for Biochemical Processes Optimization

A screening approach for non-parametric global sensitivity analysis

Generalized Parameter Estimation Method for Model-Based Real Time Optimization

Identifying Key Parameters for Design Improvement in High-Dimensional Systems With Uncertainty

Random sampling-based automatic parameter tuning for nonlinear programming solvers

Attention-Based EDA Tool Parameter Explorer: from Hybrid Parameters to Multi-QoR Metrics

Sensitivity Analysis for Computationally Expensive Models using Optimization and Objective-oriented Surrogate Approximations

Asynchronous Parallel Surrogate Optimization Algorithm for Quantitative Strategy Parameter Tuning

Aerodynamic shape optimization using design-variables-screening method

Parameter Estimation Using Metaheuristics in Systems Biology: A Comprehensive Review

Process Parameters Optimization of Injection Molding Using a Fast Strip Analysis As a Surrogate Model

An adaptive multi-output Gaussian process surrogate model for large-scale parameter estimation problems

Interaction-Aware Sensitivity Analysis for Aerodynamic Optimization Results using Information Theory

Surrogate-Based Simulation Optimization

A Method for Selecting Surrogate Models in Crashworthiness Optimization

An approximate Itô-SDE based simulated annealing algorithm for multivariate design optimization problems

Understanding the effect of hyperparameter optimization on machine learning models for structure design problems

An effective parameter screening method based on sensitivity analysis

Multi-response robust parameter design based on Bayesian mixed effects model

ANOVA-GP Modeling for High-Dimensional Bayesian Inverse Problems