Abstract:With the advent of massive data sets, much of the computational science and engineering community has moved toward data-intensive approaches in regression and classification. However, these present significant challenges due to increasing size, complexity, and dimensionality of the problems. In particular, covariance matrices in many cases are numerically unstable, and linear algebra shows that often such matrices cannot be inverted accurately on a finite precision computer. A common ad hoc approach to stabilizing a matrix is application of a so-called nugget. However, this can change the model and introduce error to the original solution. It is well known from numerical analysis that ill-conditioned matrices cannot be accurately inverted. In this paper, we develop a multilevel computational method that scales well with the number of observations and dimensions. A multilevel basis is constructed adapted to a kd-tree partitioning of the observations. Numerically unstable covariance matrices with large condition numbers can be transformed into well-conditioned multilevel ones without compromising accuracy. Moreover, it is shown that the multilevel prediction exactly solves the best linear unbiased predictor (BLUP) and generalized least squares (GLS) model, but is numerically stable. The multilevel method is tested on numerically unstable problems of up to 25 dimensions. Numerical results show speedups of up to 42,050 times for solving the BLUP problem, but with the same accuracy as the traditional iterative approach. For very ill-conditioned cases, the speedup is infinite. In addition, decay estimates of the multilevel covariance matrices are derived based on high dimensional interpolation techniques from the field of numerical analysis. This work lies at the intersection of statistics, uncertainty quantification, high performance computing, and computational applied mathematics.

Empirical Best Linear Unbiased Predictors in Multivariate Nested-Error Regression Models

Empirical Best Prediction for Small Area Inference with Binary Data

Robust small area estimation for unit level model with density power divergence

On estimation of mean squared errors of benchmarked empirical Bayes estimators

Impact of existence and nonexistence of pivot on the coverage of empirical best linear prediction intervals for small areas

An Unbiased Predictor for Skewed Response Variable with Measurement Error in Covariate

Robust Bayesian Inference for Berkson and Classical Measurement Error Models

Inference for High-Dimensional Linear Expectile Regression with De-Biasing Method

Spatial best linear unbiased prediction: a computational mathematics approach for high dimensional massive datasets

Robust Bayesian small area estimation using the sub-Gaussian -stable distribution for measurement error in covariates

Small Area Poverty Estimation under Heteroskedasticity

Semi-Parametric Empirical Best Prediction for small area estimation of unemployment indicators

Alternative Mean Square Error Estimators and Confidence Intervals for Prediction of Nonlinear Small Area Parameters

Uniform Post Selection Inference for LAD Regression and Other Z-estimation problems

Estimators for multivariate allometric regression model

Unifying area and unit-level small area estimation through calibration

A Hierarchical Bayes Unit-Level Small Area Estimation Model for Normal Mixture Populations

Transformed Fay-Herriot Model with Measurement Error in Covariates

Multivariate, Heteroscedastic Empirical Bayes via Nonparametric Maximum Likelihood

A High-dimensional M-estimator Framework for Bi-level Variable Selection

Efficient Estimation for Error Component Seemingly Unrelated Nonparametric Regression Models