A Sequential Design Approach for Calibrating Dynamic Computer Simulators

Ru Zhang,C. Devon Lin,Pritam Ranjan

DOI: https://doi.org/10.1137/18M1224544

2019-11-26

Abstract:Computer simulators are widely used to describe and explore complex physical processes. The simulator outputs may come in various formats: scalar, multivariate, functional, time series, spatial tem...

mathematics, interdisciplinary applications,physics, mathematical

What problem does this paper attempt to address?

Direct Simulation for CAD Models Undergoing Parametric Modifications

Liangchao Zhu,Ming Li,Ralph R. Martin

DOI: https://doi.org/10.1016/j.cad.2016.05.006

IF: 3.652

2016-01-01

Computer-Aided Design

Abstract:We propose a novel approach—direct simulation—for interactive simulation with accuracy control, for CAD models undergoing parametric modifications which leave Dirichlet boundary conditions unchanged. This is achieved by computing offline a generic solution as a function of the design modification parameters. Using this parametric expression, each time the model parameters are edited, the associated simulation solution for this model instance can be cheaply and quickly computed online by evaluating the derived parametric solution for these parameter values. The proposed approach furthermore works for models undergoing topological changes, and does not need any mesh regeneration or mesh mapping. These results are achieved by use of the proper generalized decomposition model reduction technique, in combination with R-functions. We believe this is the first approach that can interactively simulate the physical properties of a CAD model, even undergoing topological change, without expensive re-computation. The approach is demonstrated for linear elasticity analysis; numerical results demonstrate its simulation accuracy and efficiency in comparison with the classic FE method.
Design of SIMUPRO series training simulators for large scale industrial processes

Jiangang Lu

DOI: https://doi.org/10.1109/WCICA.2004.1343132

2004-01-01

Abstract:Training simulator can model all phenomena of startup, shutdown, routine operation and malfunction in actual factory without any raw materials and danger. An operator trained by training simulator for several weeks can get more experiences than one working in the field for two to five years. As a scientific training method, training simulator is attracting more and more considerable attentions. SIMUPRO series training simulators for large scale industrial processes developed by Zhejiang University, have been successfully applied in many petrochemical works where distributed control systems (DCS) are used. SIMUPRO series training simulators are remarked well by broad users for their advanced technology, practicability and reliability. The functions, architectures, designs and characters of these training simulators are elucidated in this work.
Automated Calibration of Parallel and Distributed Computing Simulators: A Case Study

Jesse McDonald,Maximilian Horzela,Frédéric Suter,Henri Casanova

2024-07-02

Abstract:Many parallel and distributed computing research results are obtained in simulation, using simulators that mimic real-world executions on some target system. Each such simulator is configured by picking values for parameters that define the behavior of the underlying simulation models it implements. The main concern for a simulator is accuracy: simulated behaviors should be as close as possible to those observed in the real-world target system. This requires that values for each of the simulator's parameters be carefully picked, or "calibrated," based on ground-truth real-world executions. Examining the current state of the art shows that simulator calibration, at least in the field of parallel and distributed computing, is often undocumented (and thus perhaps often not performed) and, when documented, is described as a labor-intensive, manual process. In this work we evaluate the benefit of automating simulation calibration using simple algorithms. Specifically, we use a real-world case study from the field of High Energy Physics and compare automated calibration to calibration performed by a domain scientist. Our main finding is that automated calibration is on par with or significantly outperforms the calibration performed by the domain scientist. Furthermore, automated calibration makes it straightforward to operate desirable trade-offs between simulation accuracy and simulation speed.

Distributed, Parallel, and Cluster Computing,Performance
Bayesian Adaptive Calibration and Optimal Design

Rafael Oliveira,Dino Sejdinovic,David Howard,Edwin Bonilla

2024-05-23

Abstract:The process of calibrating computer models of natural phenomena is essential for applications in the physical sciences, where plenty of domain knowledge can be embedded into simulations and then calibrated against real observations. Current machine learning approaches, however, mostly rely on rerunning simulations over a fixed set of designs available in the observed data, potentially neglecting informative correlations across the design space and requiring a large amount of simulations. Instead, we consider the calibration process from the perspective of Bayesian adaptive experimental design and propose a data-efficient algorithm to run maximally informative simulations within a batch-sequential process. At each round, the algorithm jointly estimates the parameters of the posterior distribution and optimal designs by maximising a variational lower bound of the expected information gain. The simulator is modelled as a sample from a Gaussian process, which allows us to correlate simulations and observed data with the unknown calibration parameters. We show the benefits of our method when compared to related approaches across synthetic and real-data problems.

Machine Learning
Simulation experiment design for calibration via active learning

Özge Sürer

DOI: https://doi.org/10.1080/00224065.2024.2391780

IF: 2.182

2024-10-11

Journal of Quality Technology

Abstract:Simulation models often have parameters as input and return outputs to understand the behavior of complex systems. Calibration is the process of estimating the values of the parameters in a simulation model in light of observed data from the system that is being simulated. When simulation models are expensive, emulators are built with simulation data as a computationally efficient approximation of an expensive model. An emulator then can be used to predict model outputs, instead of repeatedly running an expensive simulation model during the calibration process. Sequential design with an intelligent selection criterion can guide the process of collecting simulation data to build an emulator, making the calibration process more efficient and effective. This article proposes two novel criteria for sequentially acquiring new simulation data in an active learning setting by considering uncertainties on the posterior density of parameters. Analysis of several simulation experiments and real-data simulation experiments from epidemiology demonstrates that proposed approaches result in improved posterior and field predictions.

engineering, industrial,operations research & management science,statistics & probability
Physical Design using Differentiable Learned Simulators

Kelsey R. Allen,Tatiana Lopez-Guevara,Kimberly Stachenfeld,Alvaro Sanchez-Gonzalez,Peter Battaglia,Jessica Hamrick,Tobias Pfaff

DOI: https://doi.org/10.48550/arXiv.2202.00728

2022-02-02

Abstract:Designing physical artifacts that serve a purpose - such as tools and other functional structures - is central to engineering as well as everyday human behavior. Though automating design has tremendous promise, general-purpose methods do not yet exist. Here we explore a simple, fast, and robust approach to inverse design which combines learned forward simulators based on graph neural networks with gradient-based design optimization. Our approach solves high-dimensional problems with complex physical dynamics, including designing surfaces and tools to manipulate fluid flows and optimizing the shape of an airfoil to minimize drag. This framework produces high-quality designs by propagating gradients through trajectories of hundreds of steps, even when using models that were pre-trained for single-step predictions on data substantially different from the design tasks. In our fluid manipulation tasks, the resulting designs outperformed those found by sampling-based optimization techniques. In airfoil design, they matched the quality of those obtained with a specialized solver. Our results suggest that despite some remaining challenges, machine learning-based simulators are maturing to the point where they can support general-purpose design optimization across a variety of domains.

Machine Learning
Solving an Inverse Problem for Time Series Valued Computer Simulators via Multiple Contour Estimation

Pritam Ranjan,Joseph Resch,Abhyuday Mandal

DOI: https://doi.org/10.48550/arXiv.2211.01119

2022-11-02

Abstract:Computer simulators are often used as a substitute of complex real-life phenomena which are either expensive or infeasible to experiment with. This paper focuses on how to efficiently solve the inverse problem for an expensive to evaluate time series valued computer simulator. The research is motivated by a hydrological simulator which has to be tuned for generating realistic rainfall-runoff measurements in Athens, Georgia, USA. Assuming that the simulator returns g(x,t) over L time points for a given input x, the proposed methodology begins with a careful construction of a discretization (time-) point set (DPS) of size $k << L$, achieved by adopting a regression spline approximation of the target response series at k optimal knots locations $\{t^*_1, t^*_2, ..., t^*_k\}$. Subsequently, we solve k scalar valued inverse problems for simulator $g(x,t^*_j)$ via the contour estimation method. The proposed approach, named MSCE, also facilitates the uncertainty quantification of the inverse solution. Extensive simulation study is used to demonstrate the performance comparison of the proposed method with the popular competitors for several test-function based computer simulators and a real-life rainfall-runoff measurement model.

Methodology,Computation
Simulation Model Calibration with Dynamic Stratification and Adaptive Sampling

Pranav Jain,Sara Shashaani,Eunshin Byon

2024-07-16

Abstract:Calibrating simulation models that take large quantities of multi-dimensional data as input is a hard simulation optimization problem. Existing adaptive sampling strategies offer a methodological solution. However, they may not sufficiently reduce the computational cost for estimation and solution algorithm's progress within a limited budget due to extreme noise levels and heteroskedasticity of system responses. We propose integrating stratification with adaptive sampling for the purpose of efficiency in optimization. Stratification can exploit local dependence in the simulation inputs and outputs. Yet, the state-of-the-art does not provide a full capability to adaptively stratify the data as different solution alternatives are evaluated. We devise two procedures for data-driven calibration problems that involve a large dataset with multiple covariates to calibrate models within a fixed overall simulation budget. The first approach dynamically stratifies the input data using binary trees, while the second approach uses closed-form solutions based on linearity assumptions between the objective function and concomitant variables. We find that dynamical adjustment of stratification structure accelerates optimization and reduces run-to-run variability in generated solutions. Our case study for calibrating a wind power simulation model, widely used in the wind industry, using the proposed stratified adaptive sampling, shows better-calibrated parameters under a limited budget.

Methodology,Probability,Computation
A Novel Global Model Calibration Scheme for Complex Process Systems Using an Adaptive Experimental Design

Yao Tong,Zhijiang Shao

DOI: https://doi.org/10.23919/CCC58697.2023.10240925

2023-01-01

Abstract:First principle modeling is a crucial step before implementing model-based control and optimization of large-scale complex process systems. Due to lack of prior knowledge of process devices and technologies, the analysis of first principle models first requires tackling the calibration problem of uncertain parameters, especially when the model parameters change with operating conditions of the system. To ensure the extrapolation accuracy of model over the global operating domain and meanwhile reduce the experimental cost of condition testing, this work presents a global model calibration scheme based on an adaptive experimental design. The experimental conditions are selected by criteria of sampling density, local nonlinearity and parameter uncertainty to perform parameter estimation. To prove the effectiveness of our scheme, the algorithm in this work is applied to the case of high-temperature gas-cooled reactor pebble-bed module nuclear power plant and compared with some conventional experimental designs.
A semi-empirical approach to calibrate simulation models for semiconductor devices

Rahul Jaiswal,Manel Martínez-Ramón,Tito Busani

DOI: https://doi.org/10.1038/s41598-023-36196-z

2023-06-27

Abstract:Semiconductor device optimization using computer-based prototyping techniques like simulation or machine learning digital twins can be time and resource efficient compared to the conventional strategy of iterating over device design variations by fabricating the actual device. Ideally, simulation models require perfect calibration of material parameters for the model to represent a particular semiconductor device. This calibration process itself can require characterization information of the device and its precursors and extensive expert knowledge of non characterizable parameters and their tuning. We propose a hybrid method to calibrate multiple simulation models for a device using minimal characterization data and machine learning-based prediction models. A photovoltaic device is chosen as the example for this technique where optical and electrical simulation models of an industrially manufactured silicon solar cell are calibrated and the simulated device performance is compared with the measurement data from the physical device.
Simulation and Optimization of Chemical Process Using GAMS Mix Pattern

Hao Li,Zhijiang Shao

DOI: https://doi.org/10.1109/wcica.2006.1713296

2006-01-01

Abstract:Rigorous simulation and optimization of chemical process systems usually is based on a realistic calculation of the physical properties. Within GAMS models, nonlinear functions for the realistic calculation leads to very complex models and greatly increased dimensions. By using the new mechanism, a mix pattern with external functions, the situation can be improved by transferring calculation procedures to an external module. In this work, the external functions, written in the programming language Fortran, are used for the physical properties calculation in the flash process of a distillation column. The accuracy of the result is comparable to standard simulation software
Dynamic Process Calibration Based on Sparse Partial Least Squares

Qiaojun Wen,Zhiqiang Ge,Zhihuan Song,Peiliang Wang

DOI: https://doi.org/10.1109/wcica.2014.7052918

2014-01-01

Abstract:This article proposes a sparse partial least squares (SPLS) for model calibration of dynamic processes. Via capturing the relationship of process inputs and measurements at different sampling instances, partial least squares (PLS) is a typical multivariable statistical process control technique to model dynamic processes. However, due to rare process measurements, large number of process variables and large time scale of process dynamics, the over-fitting problem will be obvious and the calibration performance will be degraded. With the sparse representation, SPLS produces a more reliable model to capture the process dynamics, which won't be deteriorated by the small sample size problem. Case studies on a simulation example and the Tennessee Eastman (TE) process illustrate the effectiveness of the proposed method.
Robust Experimental Designs for Model Calibration

Arvind Krishna,V. Roshan Joseph,Shan Ba,William A. Brenneman,William R. Myers

DOI: https://doi.org/10.1080/00224065.2021.1930618

2020-08-03

Abstract:A computer model can be used for predicting an output only after specifying the values of some unknown physical constants known as calibration parameters. The unknown calibration parameters can be estimated from real data by conducting physical experiments. This paper presents an approach to optimally design such a physical experiment. The problem of optimally designing physical experiment, using a computer model, is similar to the problem of finding optimal design for fitting nonlinear models. However, the problem is more challenging than the existing work on nonlinear optimal design because of the possibility of model discrepancy, that is, the computer model may not be an accurate representation of the true underlying model. Therefore, we propose an optimal design approach that is robust to potential model discrepancies. We show that our designs are better than the commonly used physical experimental designs that do not make use of the information contained in the computer model and other nonlinear optimal designs that ignore potential model discrepancies. We illustrate our approach using a toy example and a real example from industry.

Methodology,Applications
Design and Implementation of Reconfigurational Data Simulator

LI Wen-xin,LEI Zhi-guang,MU De-jun

DOI: https://doi.org/10.3969/j.issn.1000-3428.2011.19.075

2011-01-01

Abstract:With the purpose to support debugging,unit testing,system test and troubleshooting on a craft informational terminal,data simulators are always required.A reconfigurational architecture of simulator is presented through study reconfigurational technology.Reconfigurational data simulator is designed by combining databases and interpreters.In the end,several data simulators are implemented for different requirements via utilizing hardware and software reconfigurational technology.All these data simulations are applied successfully and the related results show that data simulators have the features of high stability in performance,flexibility in configuration and strong reconfigurability in functions.
Data assimilation for online model calibration in discrete event simulation

Xiaolin Hu,Mingxi Yan

DOI: https://doi.org/10.1177/00375497231221578

2024-01-14

SIMULATION

Abstract:SIMULATION, Ahead of Print. The increasing availability of real-time data collected from dynamic systems brings opportunities for simulation models to be calibrated online for improving the accuracy of simulation-based studies. Systematical methods are needed for assimilating real-time measurement data into simulation models. This paper presents a particle filter-based data assimilation method to support online model calibration in discrete event simulation. A joint state-parameter estimation problem is defined, and a particle filter-based data assimilation algorithm is presented. The developed method is applied to a discrete event simulation of a one-way traffic control system. Experiments results demonstrate the effectiveness of the developed method for calibrating simulation models' parameters in real time and for improving data assimilation results.

computer science, interdisciplinary applications, software engineering
Computer Model Calibration with Time Series Data using Deep Learning and Quantile Regression

Saumya Bhatnagar,Won Chang,Seonjin Kim Jiali Wang

DOI: https://doi.org/10.48550/arXiv.2008.13066

2020-09-08

Abstract:Computer models play a key role in many scientific and engineering problems. One major source of uncertainty in computer model experiment is input parameter uncertainty. Computer model calibration is a formal statistical procedure to infer input parameters by combining information from model runs and observational data. The existing standard calibration framework suffers from inferential issues when the model output and observational data are high-dimensional dependent data such as large time series due to the difficulty in building an emulator and the non-identifiability between effects from input parameters and data-model discrepancy. To overcome these challenges we propose a new calibration framework based on a deep neural network (DNN) with long-short term memory layers that directly emulates the inverse relationship between the model output and input parameters. Adopting the 'learning with noise' idea we train our DNN model to filter out the effects from data model discrepancy on input parameter inference. We also formulate a new way to construct interval predictions for DNN using quantile regression to quantify the uncertainty in input parameter estimates. Through a simulation study and real data application with WRF-hydro model we show that our approach can yield accurate point estimates and well calibrated interval estimates for input parameters.

Machine Learning,Methodology
An Approach for Realistically Simulating the Performance of Scientific Applications on High Performance Computing Systems

Ali Mohammed,Ahmed Eleliemy,Florina M. Ciorba,Franziska Kasielke,Ioana Banicescu

DOI: https://doi.org/10.48550/arXiv.1910.06844

2019-10-15

Abstract:Scientific applications often contain large, computationally-intensive, and irregular parallel loops or tasks that exhibit stochastic characteristics. Applications may suffer from load imbalance during their execution on high-performance computing (HPC) systems due to such characteristics. Dynamic loop self-scheduling (DLS) techniques are instrumental in improving the performance of scientific applications on HPC systems via load balancing. Selecting a DLS technique that results in the best performance for different problems and system sizes requires a large number of exploratory experiments. A theoretical model that can be used to predict the scheduling technique that yields the best performance for a given problem and system has not yet been identified. Therefore, simulation is the most appropriate approach for conducting such exploratory experiments with reasonable costs. This work devises an approach to realistically simulate computationally-intensive scientific applications that employ DLS and execute on HPC systems. Several approaches to represent the application tasks (or loop iterations) are compared to establish their influence on the simulative application performance. A novel simulation strategy is introduced, which transforms a native application code into a simulative code. The native and simulative performance of two computationally-intensive scientific applications are compared to evaluate the realism of the proposed simulation approach. The comparison of the performance characteristics extracted from the native and simulative performance shows that the proposed simulation approach fully captured most of the performance characteristics of interest. This work shows and establishes the importance of simulations that realistically predict the performance of DLS techniques for different applications and system configurations.

Distributed, Parallel, and Cluster Computing,Performance
Dynamic System Stability Verification Using Numerical Simulator

Jongrae Kim

2023-11-20

Abstract:There are recent shifts in demand for design controllers from simplified to complex model-based. Although simplification approaches are successful in many areas of engineering control systems, high-fidelity simulation-based control design, for example, reinforcement learning, has been rising in robotics areas. On the other hand, the lack of assurances about the stability and robustness of simulation-based control design restricts its applications to safety-critical systems. We develop computational methods to verify the stability and robustness of safety-critical systems. By extending the inverse Lyapunov theorem, we present a practical method to compute the constants required to check the exponential stability conditions of dynamic systems implemented in a numerical simulator. It is shown that the norm-bound of the propagated states is a function of the numerical integration steps, where the numerical simulator may include discontinuous jumps of states. The energy bounds for the transition states are obtained based on the exponential stability assumption of the inverse Lyapunov theorem. Finally, a finite sampling algorithm provides the deterministic stability guarantee for the continuous state space.

Systems and Control,Dynamical Systems,Optimization and Control
A Novel Real-Time Hybrid Simulator Base On Two Personal Computers And Analog Interfaces For Data Exchange

Shuqing Zhang,Luyuan Tong,Xu Liang,Chao Hong

DOI: https://doi.org/10.1109/PES.2009.5275991

2009-01-01

Abstract:For power system simulation, the contradiction between the computational efficiency and the system scale as well as the detailed modeling comes into being. The electromagnetic / electromechanical hybrid simulation technology has been thereupon proposed. In this paper, a novel real-time hybrid simulator, which is based on two personal computers (PCs) and an analog communication device for data exchange between two PCs, is designed and built, and the key techniques are studied. For the unsolved essential difficulties in hybrid simulation technology, this paper initially studies the forecasting and correction method. The case studies not only indicate that the hybrid simulator is a success, but also testify to the practicability and necessity of the forecasting and correction method. All the achievements have paved the way for the forthcoming studies on the original hybrid simulator based on digital computers and real-time digital simulator (RTDS).

A Sequential Design Approach for Calibrating Dynamic Computer Simulators

Direct Simulation for CAD Models Undergoing Parametric Modifications

Design of SIMUPRO series training simulators for large scale industrial processes

Automated Calibration of Parallel and Distributed Computing Simulators: A Case Study

Bayesian Adaptive Calibration and Optimal Design

Simulation experiment design for calibration via active learning

Physical Design using Differentiable Learned Simulators

Solving an Inverse Problem for Time Series Valued Computer Simulators via Multiple Contour Estimation

Simulation Model Calibration with Dynamic Stratification and Adaptive Sampling

A Novel Global Model Calibration Scheme for Complex Process Systems Using an Adaptive Experimental Design

A semi-empirical approach to calibrate simulation models for semiconductor devices

Simulation and Optimization of Chemical Process Using GAMS Mix Pattern

Dynamic Process Calibration Based on Sparse Partial Least Squares

Robust Experimental Designs for Model Calibration

Design and Implementation of Reconfigurational Data Simulator

Data assimilation for online model calibration in discrete event simulation

Computer Model Calibration with Time Series Data using Deep Learning and Quantile Regression

An Approach for Realistically Simulating the Performance of Scientific Applications on High Performance Computing Systems

Dynamic System Stability Verification Using Numerical Simulator

A Novel Real-Time Hybrid Simulator Base On Two Personal Computers And Analog Interfaces For Data Exchange