Conditional variational autoencoder with Gaussian process regression recognition for parametric models

Xuehan Zhang,Lijian Jiang

2023-05-17

Abstract:In this article, we present a data-driven method for parametric models with noisy observation data. Gaussian process regression based reduced order modeling (GPR-based ROM) can realize fast online predictions without using equations in the offline stage. However, GPR-based ROM does not perform well for complex systems since POD projection are naturally linear. Conditional variational autoencoder (CVAE) can address this issue via nonlinear neural networks but it has more model complexity, which poses challenges for training and tuning hyperparameters. To this end, we propose a framework of CVAE with Gaussian process regression recognition (CVAE-GPRR). The proposed method consists of a recognition model and a likelihood model. In the recognition model, we first extract low-dimensional features from data by POD to filter the redundant information with high frequency. And then a non-parametric model GPR is used to learn the map from parameters to POD latent variables, which can also alleviate the impact of noise. CVAE-GPRR can achieve the similar accuracy to CVAE but with fewer parameters. In the likelihood model, neural networks are used to reconstruct data. Besides the samples of POD latent variables and input parameters, physical variables are also added as the inputs to make predictions in the whole physical space. This can not be achieved by either GPR-based ROM or CVAE. Moreover, the numerical results show that CVAE-GPRR may alleviate the overfitting issue in CVAE.

Computational Engineering, Finance, and Science,Machine Learning

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve The paper aims to address the following issues: 1. **Modeling and Predicting Parameter Models**: - For unknown parameter models, when only noisy observational data is available, a data-driven approach is proposed to model these parameter models and make predictions within the unobserved parameter space and physical regions. 2. **Improving Prediction Accuracy and Reducing Model Complexity**: - Compared to traditional Variational Autoencoders (VAE) and Conditional Variational Autoencoders (CVAE), a framework combining Gaussian Process Regression (GPR) (CVAE-GPRR) is proposed to achieve fewer parameters while maintaining similar accuracy, thereby alleviating the overfitting problem. 3. **Handling Noisy Data in Complex Systems**: - Principal Component Analysis (PCA) is used to extract low-dimensional features from the data, and Gaussian Process Regression (GPR) is employed to establish a mapping from the parameter space to the latent space, thereby better handling noisy data. 4. **Generating Samples Across the Entire Physical Space**: - By incorporating physical variables as inputs into the likelihood model, the model can generate samples in unobserved physical regions, which is not achievable by traditional Gaussian Process Regression-based Reduced Order Models (GPR-based ROM) or standard CVAE. In summary, the main objective of the paper is to develop an efficient and robust data-driven method for modeling and predicting parameter models in the presence of noisy observational data.

Conditional variational autoencoder with Gaussian process regression recognition for parametric models

Multi-modal Gaussian Process Variational Autoencoders for Neural and Behavioral Data

Variational Autoregressive Decoder for Neural Response Generation.

Gaussian process latent variable model factorization for context-aware recommender systems

Robust and Scalable Gaussian Process Regression and Its Applications

Scalable Variational Gaussian Process Regression Networks

Variational Auto Encoder Fused with Gaussian Process for Unsupervised Anomaly Detection

The Gaussian Discriminant Variational Autoencoder (GdVAE): A Self-Explainable Model with Counterfactual Explanations

Gaussian Processes Autoencoder for Dimensionality Reduction.

Nonparametrically Guided Autoencoder with Laplace Approximation for Dimensionality Reduction.

Learning conditional variational autoencoders with missing covariates

Neural Gaussian Copula for Variational Autoencoder

Gaussian Process Latent Variable Model Based on Immune Clonal Selection for SAR Target Feature Extraction and Recognition

Conditional Gaussian Distribution Learning for Open Set Recognition

CVA2E: A Conditional Variational Autoencoder with an Adversarial Training Process for Hyperspectral Imagery Classification

Variational Inference for Uncertainty on the Inputs of Gaussian Process Models

Variational Learning of Gaussian Process Latent Variable Models through Stochastic Gradient Annealed Importance Sampling

Latent Variable Double Gaussian Process Model for Decoding Complex Neural Data

Gaussian Copula Variational Autoencoders for Mixed Data

Modeling-Learning-Based Actor-Critic Algorithm with Gaussian Process Approximator

A data-driven method for parametric PDE Eigenvalue Problems using Gaussian Process with different covariance functions