Abstract:Multi-fidelity surrogate (MFS) modeling incorporates a large number of low-fidelity (LF) samples with a small size of high-fidelity (HF) samples to obtain accurate HF approximations of unknown inputs. Because they save costs when labeling massive HF samples, MFS modeling techniques have been widely used in many engineering problems such as rapid simulation and mechanism optimization. In this paper, we propose a framework of randomized stacking with active learning for incremental MFS modeling (RSAL-iMFS). The framework contains two parts: randomized stacking for incremental MFS modeling (RS-iMFS) and query-by-committee (QBC)-based active learning for MFS modeling (QBC-AL-MFS). In particular, we first randomly project the inputs of LF samples into different spaces spanned by random projection matrices to form new LF samples and build a series of LF regressors based on the transformed LF samples to capture the LF features from different views. Then, these base LF regressors are stacked to form the inputs of the subsequent incremental Gaussian process regression (iGPR) model for approximating the HF responses. To achieve improved modeling performance, we also adopt the QBC-based active learning (QBC-AL) method to recommend unlabeled inputs from the input pool of candidate HF samples according to their distances, which are weighted by using the divergence of the committee voting results. We then label these recommended inputs to incrementally update the current iGPR model. Numerical experiments validate the proposed framework in incremental MFS modeling tasks and show that (1) RSAL-iMFS outperforms the state-of-the-art models; (2) Gaussian projection matrices (GPMs) can be reasonable choices in most cases; and (3) RSAL-iMFS’s modeling performance has low sensitivity to the choice of the number of base regressors.

A Scalable and Extensible Framework for Superposition-Structured Models

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

SEP-QN: Scalable and Extensible Proximal Quasi-Newton Method for Dirty Statistical Models

A Local Online Learning Approach for Non-linear Data.

The Substitution Secant/Finite Difference Method for Large Scale Sparse Unconstrained Optimization

Asynchronous Proximal Stochastic Gradient Algorithm for Composition Optimization Problems

Smoothing Proximal Gradient Method for General Structured Sparse Learning

An Alternating Proximal Splitting Method With Global Convergence For Nonconvex Structured Sparsity Optimization

A Unifying Framework for Convergence Analysis of Approximate Newton Methods.

Structured Sparsity Optimization with Non-Convex Surrogates of L2,0-Norm: A Unified Algorithmic Framework

A Multi-Fidelity Surrogate Modeling Approach for Incorporating Multiple Non-Hierarchical Low-Fidelity Data

Multi-block linearized alternating direction method for sparse fused Lasso modeling problems

Incorporating Linear Regression Problems into an Adaptive Framework with Feasible Optimizations.

A Novel Multi-fidelity Surrogate Model with Two-Stage Ensemble

A novel multi-fidelity surrogate modeling framework integrated with sequential sampling criterion for non-hierarchical data

Structured Sparsity Optimization with Non-Convex Surrogates of $\ell _{2,0}$ℓ2,0-Norm: A Unified Algorithmic Framework

Proximal methods for structured nonsmooth optimization over Riemannian submanifolds

A Feasible Level Proximal Point Method for Nonconvex Sparse Constrained Optimization

RSAL-iMFS: A Framework of Randomized Stacking with Active Learning for Incremental Multi-Fidelity Surrogate Modeling

Switchable Representation Learning Framework with Self-Compatibility.