Abstract:Active learning provides guidance for the design and modeling of systems with highly expensive sampling costs. However, existing active learning approaches suffer from cold-start concerns, where the performance is impaired due to the initial few experiments designed by active learning. In this paper, we propose using transfer learning to solve the cold-start problem of functional regression by leveraging knowledge from related and data-rich signals to achieve robust and superior performance, especially when only a few experiments are available in the signal of interest. More specifically, we construct a multi-output Gaussian process (MGP) to model the between-signal functional relationship. This MGP features unique innovations that distinguish the proposed transfer active learning from existing works: i) a specially designed covariance structure is proposed for characterizing within-and between-signal inter-relationships and facilitating interpretable transfer learning, and ii) an iterative Bayesian framework is proposed to update the parameters and prediction of the MGP in real-time, which significantly reduces the computational load and facilitates the iterative active learning. The inter-relationship captured by this novel MGP is then fed into active learning using the integrated mean-squared error (IMSE) as the objective. We provide theoretical justifications for this active learning mechanism, which demonstrates the objective (IMSE) is monotonically decreasing as we gather more data from the proposed transfer active learning. The real-time updating and the monotonically decreasing objective together provide both practical efficiency and theoretical guarantees for solving the cold-start concern in active learning. The proposed method is compared with benchmark methods through various numerical and real case studies, and the results demonstrate the superiority of the method, especially when limited experiments are available at the initial stage of design.

Real-Time Transfer Active Learning for Functional Regression and Prediction Based on Multi-Output Gaussian Process

Transfer Learning Based on Incorporating Source Knowledge Using Gaussian Process Models for Quick Modeling of Dynamic Target Processes

Short-term Traffic Prediction with Deep Neural Networks and Adaptive Transfer Learning

Amortized Active Learning for Nonparametric Functions

Local transfer learning Gaussian process modeling, with applications to surrogate modeling of expensive computer simulators

Regularized Multi-Output Gaussian Convolution Process With Domain Adaptation

An Expected Integrated Error Reduction Function for Accelerating Bayesian Active Learning of Failure Probability

Active Learning Methods with Deep Gaussian Processes.

Fast Transfer Gaussian Process Regression with Large-Scale Sources

Large Linear Multi-output Gaussian Process Learning

Scalable Multi-Task Gaussian Processes with Neural Embedding of Coregionalization

RSAL-iMFS: A Framework of Randomized Stacking with Active Learning for Incremental Multi-Fidelity Surrogate Modeling

Promoting Active Learning with Mixtures of Gaussian Processes

Non-stationary and Sparsely-correlated Multi-output Gaussian Process with Spike-and-Slab Prior

A Deep Generative Active Learning-based Scheme for the Efficient Development of Transient Stability Predictor

Iterative self-transfer learning: A general methodology for response time-history prediction based on small dataset

Near-Optimal Active Learning of Multi-Output Gaussian Processes

Adaptive Transfer Kernel Learning for Transfer Gaussian Process Regression

Dynamic Transfer Gaussian Process Regression.

Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes