Abstract:We investigate the use of derivative information for Batch Active Learning in Gaussian Process regression models. The proposed approach employs the predictive covariance matrix for selection of data batches to exploit full correlation of samples. We theoretically analyse our proposed algorithm taking different optimality criteria into consideration and provide empirical comparisons highlighting the advantage of incorporating derivatives information. Our results show the effectiveness of our approach across diverse applications.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to use derivative information for Batch Active Learning (BAL) in the Gaussian process regression model. Specifically, the authors propose a new method to fully utilize the complete correlation between samples by selecting data batches through the predictive covariance matrix. They aim to prove the effectiveness and advantages of introducing derivative information in the batch active learning process, and this has been proven through theoretical analysis and experiments. ### Main problems 1. **How to use derivative information**: The paper explores how to use derivative information in batch active learning of Gaussian process regression to improve the learning efficiency and accuracy of the model. 2. **Optimization selection criteria**: The authors study different optimization criteria (such as D - optimality, A - optimality and E - optimality), and provide theoretical analysis to illustrate the performance of these criteria after introducing derivative information. 3. **Effectiveness in practical applications**: The paper verifies the effectiveness and superiority of the proposed method in different application scenarios through multiple experiments, including simulated functions, industrial systems and geographical data reconstruction. ### Theoretical contributions - **Information gain analysis**: By defining Information Gain, the authors prove that introducing derivative information can provide more information, thereby improving the learning effect. - **Decay rate of the predictive covariance matrix**: The authors show the decay rate of the predictive covariance matrix when using derivative information and prove that it is superior to the situation without using derivative information. ### Experimental verification - **Simulated function**: Through an experiment on a two - dimensional heart - shaped function, it is shown that the model can better capture the function characteristics after introducing derivative information. - **High - pressure fuel supply system**: The proposed method is applied in an industrial scenario to verify its effectiveness and safety in dynamic non - linear systems. - **Map reconstruction**: Through the geographical data reconstruction experiment, the superior performance of the method in practical applications is shown. In general, the main purpose of this paper is to improve the performance of batch active learning in Gaussian process regression by introducing derivative information, and the effectiveness of this improvement has been proven through theory and experiments.

Batch Active Learning in Gaussian Process Regression using Derivatives

Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes

Scaling Gaussian Process Regression with Derivatives

DGP-LVM: Derivative Gaussian process latent variable models

Batched Gaussian Process Bandit Optimization via Determinantal Point Processes

Black-Box Batch Active Learning for Regression

Gaussian process learning of nonlinear dynamics

A Mini-Batch Method for Solving Nonlinear PDEs with Gaussian Processes

Active Learning for Abrupt Shifts Change-point Detection via Derivative-Aware Gaussian Processes

Exploiting gradients and Hessians in Bayesian optimization and Bayesian quadrature

Active Learning for Gaussian Process Considering Uncertainties with Application to Shape Control of Composite Fuselage

Epoch-Evolving Gaussian Process Guided Learning for Classification

A Solution to the Ill-Conditioning of Gradient-Enhanced Covariance Matrices for Gaussian Processes

Natural Gradients in Practice: Non-Conjugate Variational Inference in Gaussian Process Models

Adaptive Batch Sizes for Active Learning A Probabilistic Numerics Approach

Stochastic Gradient Descent for Gaussian Processes Done Right

Dual Control with Active Learning using Gaussian Process Regression

Active Learning with Weak Supervision for Gaussian Processes

Hierarchical-Hyperplane Kernels for Actively Learning Gaussian Process Models of Nonstationary Systems

Global Optimization of Gaussian processes

Asynchronous Parallel Large-Scale Gaussian Process Regression