Abstract:Metamodelling is an increasingly more popular approach for alleviating the computational burden associated with computationally intensive optimization/management problems in environmental and water resources systems. Some studies refer to the metamodelling approach as function approximation, surrogate modelling, response surface methodology or model emulation. A metamodel-enabled optimizer approximates the objective (or constraint) function in a way that eliminates the need to always evaluate this function via a computationally expensive simulation model. There is a sizeable body of literature developing and applying a variety of metamodelling strategies to various environmental and water resources related problems including environmental model calibration, water resources systems analysis and management, and water distribution network design and optimization. Overall, this literature generally implies metamodelling yields enhanced solution efficiency and (almost always) effectiveness of computationally intensive optimization problems. This paper initially develops a comparative assessment framework which presents a clear computational budget dependent definition for the success/failure of the metamodelling strategies, and then critically evaluates metamodelling strategies, through numerical experiments, against other common optimization strategies not involving metamodels. Three different metamodel-enabled optimizers involving radial basis functions, kriging, and neural networks are employed. A robust numerical assessment within different computational budget availability scenarios is conducted over four test functions commonly used in optimization as well as two real-world computationally intensive optimization problems in environmental and water resources systems. Numerical results show that metamodelling is not always an efficient and reliable approach to optimizing computationally intensive problems. For simpler response surfaces, metamodelling can be very efficient and effective. However, in some cases, and in particular for complex response surfaces when computational budget is not very limited, metamodelling can be misleading and a hindrance, and better solutions are achieved with optimizers not involving metamodels. Results also demonstrate that neural networks are not appropriate metamodelling tools for limited computational budgets while metamodels employing kriging and radial basis functions show comparable overall performance when the available computational budget is very limited.

Interactive effects of hyperparameter optimization techniques and data characteristics on the performance of machine learning algorithms for building energy metamodeling

Understanding the effect of hyperparameter optimization on machine learning models for structure design problems

Optimizing Machine Learning Algorithms for Landslide Susceptibility Mapping along the Karakoram Highway, Gilgit Baltistan, Pakistan: A Comparative Study of Baseline, Bayesian, and Metaheuristic Hyperparameter Optimization Techniques

Metamodeling-based simulation optimization in manufacturing problems: a comparative study

Comparing hyperparameter tuning methods in machine learning based urban building energy modeling: A study in Chicago

Efficient hyperparameters optimization through model-based reinforcement learning with experience exploiting and meta-learning

Optimization on selecting XGBoost hyperparameters using meta‐learning

A comparison of six metamodeling techniques applied to multi building performance vectors prediction on gymnasiums under multiple climate conditions

Data-mechanism-driven Product Performance Optimization with Multiple Parameters under Uncertainties in Manufacturing Automation Systems

Numerical assessment of metamodelling strategies in computationally intensive optimization

Predicting daily heating energy consumption in residential buildings through integration of random forest model and meta-heuristic algorithms

A meta-model-based optimization approach for fast and reliable calibration of building energy models

Data driven model improved by multi-objective optimisation for prediction of building energy loads

Hyperparameter Tuning Algorithm Comparison with Machine Learning Algorithms

A comprehensive comparison among metaheuristics (MHs) for geohazard modeling using machine learning: Insights from a case study of landslide displacement prediction

On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice

Hyperparameter Optimization for Machine Learning Models Based on Bayesian Optimization

Testing the Efficacy of Hyperparameter Optimization Algorithms in Short-Term Load Forecasting

On the Benefits of Using Metaheuristics in the Hyperparameter Tuning of Deep Learning Models for Energy Load Forecasting

Hyperparameter Tuning of Load-Forecasting Models Using Metaheuristic Optimization Algorithms—A Systematic Review