Abstract:In this paper we study the computation of the nonparametric maximum likelihood estimator (NPMLE) in multivariate mixture models. Our first approach discretizes this infinite dimensional convex optimization problem by fixing the support points of the NPMLE and optimizing over the mixture proportions. In this context we propose, leveraging the sparsity of the solution, an efficient and scalable semismooth Newton based augmented Lagrangian method (ALM). Our algorithm beats the state-of-the-art methods~\cite{koenker2017rebayes, kim2020fast} and can handle $n \approx 10^6$ data points with $m \approx 10^4$ support points. Our second procedure, which combines the expectation-maximization (EM) algorithm with the ALM approach above, allows for joint optimization of both the support points and the probability weights. For both our algorithms we provide formal results on their (superlinear) convergence properties. The computed NPMLE can be immediately used for denoising the observations in the framework of empirical Bayes. We propose new denoising estimands in this context along with their consistent estimates. Extensive numerical experiments are conducted to illustrate the effectiveness of our methods. In particular, we employ our procedures to analyze two astronomy data sets: (i) Gaia-TGAS Catalog~\cite{anderson2018improving} containing $n \approx 1.4 \times 10^6$ data points in two dimensions, and (ii) the $d=19$ dimensional data set from the APOGEE survey~\cite{majewski2017apache} with $n \approx 2.7 \times 10^4$.

Online Parameter Optimization Scheme for the Kernel Function-Based Mixture Models Disturbed by Colored Noises

Identification of nonlinearly parameterized nonlinear models: application to mass balance systems.

Maximum likelihood least squares‐based iterative methods for output‐error bilinear‐parameter models with colored noises

Novel parameter estimation method for the systems with colored noises by using the filtering identification idea

Kernel Least-Mean Mixed-Norm Algorithm

Mt-Omp For Hyperspectral Imagery Denoising With Model Parameter Estimation

Generalized Parameter Estimation Method for Model-Based Real Time Optimization

Decomposition‐based maximum likelihood gradient iterative algorithm for multivariate systems with colored noise

Robust Gradient Estimation Algorithm for a Stochastic System with Colored Noise

Mixture Bayesian Regularization of PCR Model and Soft Sensing Application

Robust Supervised Probabilistic Principal Component Analysis Model for Soft Sensing of Key Process Variables

Nonlinear Variational Bayesian Student’s-t Mixture Regression and Inferential Sensor Application with Semisupervised Data

Parameterized Nonlinear Least Squares for Unsupervised Nonlinear Spectral Unmixing.

Parameter Estimation Method for Generalized Time-varying Systems With Colored Noise Based on the Hierarchical Principle

Auxiliary Model-based Continuous Mixed p-norm Algorithm for Output-error Moving Average Systems Using the Multi-innovation Optimization

Adaptive Noise Covariance Estimation under Colored Noise using Dynamic Expectation Maximization

Joint iterative state and parameter estimation for bilinear systems with autoregressive noises via the data filtering

Design of the Modified Fractional Central Difference Kalman Filters under Stochastic Colored Noises.

On Efficient and Scalable Computation of the Nonparametric Maximum Likelihood Estimator in Mixture Models

Adaptive Bayesian nonparametric regression using kernel mixture of polynomials with application to partial linear model

Model transformation based distributed stochastic gradient algorithm for multivariate output-error systems