Abstract:In this paper we study the computation of the nonparametric maximum likelihood estimator (NPMLE) in multivariate mixture models. Our first approach discretizes this infinite dimensional convex optimization problem by fixing the support points of the NPMLE and optimizing over the mixture proportions. In this context we propose, leveraging the sparsity of the solution, an efficient and scalable semismooth Newton based augmented Lagrangian method (ALM). Our algorithm beats the state-of-the-art methods~\cite{koenker2017rebayes, kim2020fast} and can handle $n \approx 10^6$ data points with $m \approx 10^4$ support points. Our second procedure, which combines the expectation-maximization (EM) algorithm with the ALM approach above, allows for joint optimization of both the support points and the probability weights. For both our algorithms we provide formal results on their (superlinear) convergence properties. The computed NPMLE can be immediately used for denoising the observations in the framework of empirical Bayes. We propose new denoising estimands in this context along with their consistent estimates. Extensive numerical experiments are conducted to illustrate the effectiveness of our methods. In particular, we employ our procedures to analyze two astronomy data sets: (i) Gaia-TGAS Catalog~\cite{anderson2018improving} containing $n \approx 1.4 \times 10^6$ data points in two dimensions, and (ii) the $d=19$ dimensional data set from the APOGEE survey~\cite{majewski2017apache} with $n \approx 2.7 \times 10^4$.

New estimation in mixture of experts models using the Pearson type VII distribution

Bayesian estimation and prediction for certain mixtures

Model-based clustering and classification using mixtures of multivariate skewed power exponential distributions

On Parameter Estimation in Deviated Gaussian Mixture of Experts

A Nonparametric Estimation Method for the Multivariate Mixture Models

Semiparametric Mixture Regression with Unspecified Error Distributions

Maximum Smoothed Likelihood Estimation for a Class of Semiparametric Pareto Mixture Densities

Robust Fitting of Mixture Models using Weighted Complete Estimating Equations

Maximum-penalized-likelihood estimation for independent and Markov-dependent mixture models.

Penalized Maximum Likelihood Estimator for Mixture of von Mises-Fisher Distributions

Estimation and Hypothesis-Testing in Finite Mixture-Models

A novel approach for parameter estimation of mixture of two Weibull distributions in failure data modeling

A Generalized Finite Mixture Model for Asymmetric Probability Distribution Observations

Model Selection for Exponential Power Mixture Regression Models

A novel finite mixture model based on the generalized scale mixtures of asymmetric generalized normal distributions: properties, estimation methodology and applications

On the Behavior of the Expectation-Maximization Algorithm for Mixture Models

An Introduction to the Practical and Theoretical Aspects of Mixture-of-Experts Modeling

Estimating the Number of Components in Finite Mixture Models via Variational Approximation

Sequential Estimation for Mixture of Regression Models for Heterogeneous Population

On Efficient and Scalable Computation of the Nonparametric Maximum Likelihood Estimator in Mixture Models

Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts