Abstract:In this manuscript, we propose to use a variational autoencoder-based framework for parameterizing a conditional linear minimum mean squared error estimator. The variational autoencoder models the underlying unknown data distribution as conditionally Gaussian, yielding the conditional first and second moments of the estimand, given a noisy observation. The derived estimator is shown to approximate the minimum mean squared error estimator by utilizing the variational autoencoder as a generative prior for the estimation problem. We propose three estimator variants that differ in their access to ground-truth data during the training and estimation phases. The proposed estimator variant trained solely on noisy observations is particularly noteworthy as it does not require access to ground-truth data during training or estimation. We conduct a rigorous analysis by bounding the difference between the proposed and the minimum mean squared error estimator, connecting the training objective and the resulting estimation performance. Furthermore, the resulting bound reveals that the proposed estimator entails a bias-variance tradeoff, which is well-known in the estimation literature. As an example application, we portray channel estimation, allowing for a structured covariance matrix parameterization and low-complexity implementation. Nevertheless, the proposed framework is not limited to channel estimation but can be applied to a broad class of estimation problems. Extensive numerical simulations first validate the theoretical analysis of the proposed variational autoencoder-based estimators and then demonstrate excellent estimation performance compared to related classical and machine learning-based state-of-the-art estimators.

Automatic Regularization for Linear MMSE Filters

Regularized Least-Mean-Square Algorithms

Regularized Kernel Forms of Minimum Squared Error Method

Approximate MMSE Estimator for Linear Dynamic Systems With Gaussian Mixture Noise

Regularization Using Monte Carlo Simulation to Make Optimal Beamformers Robust to System Perturbations

Minimum Mse Based Regularization For System Identification In The Presence Of Input And Output Noise

Regularization of linear ill-posed problems involving multiplication operators

New Normalized LMS Adaptive Filter with a Variable Regularization Factor

Regularized EM algorithm

Regularization approaches for support vector machines with applications to biomedical data

Regularization by deep learning in signal processing

Optimal Regularization for System Identification from Noisy Input and Output Signals

Leveraging Variational Autoencoders for Parameterized MMSE Estimation

Analytic MMSE Bounds in Linear Dynamic Systems with Gaussian Mixture Noise Statistics

Precise Error Analysis of Regularized M-estimators in High-dimensions

Regularization Strategies and Empirical Bayesian Learning for MKL

Regularization of linear machine learning problems

A Joint Bidiagonalization Based Algorithm for Large Scale Linear Discrete Ill-posed Problems in General-Form Regularization

Regularization properties of adversarially-trained linear regression

ℓ 2 -ℓ 0 Regularization Path Tracking Algorithms