Abstract:With the advent of massive data sets, much of the computational science and engineering community has moved toward data-intensive approaches in regression and classification. However, these present significant challenges due to increasing size, complexity, and dimensionality of the problems. In particular, covariance matrices in many cases are numerically unstable, and linear algebra shows that often such matrices cannot be inverted accurately on a finite precision computer. A common ad hoc approach to stabilizing a matrix is application of a so-called nugget. However, this can change the model and introduce error to the original solution. It is well known from numerical analysis that ill-conditioned matrices cannot be accurately inverted. In this paper, we develop a multilevel computational method that scales well with the number of observations and dimensions. A multilevel basis is constructed adapted to a kd-tree partitioning of the observations. Numerically unstable covariance matrices with large condition numbers can be transformed into well-conditioned multilevel ones without compromising accuracy. Moreover, it is shown that the multilevel prediction exactly solves the best linear unbiased predictor (BLUP) and generalized least squares (GLS) model, but is numerically stable. The multilevel method is tested on numerically unstable problems of up to 25 dimensions. Numerical results show speedups of up to 42,050 times for solving the BLUP problem, but with the same accuracy as the traditional iterative approach. For very ill-conditioned cases, the speedup is infinite. In addition, decay estimates of the multilevel covariance matrices are derived based on high dimensional interpolation techniques from the field of numerical analysis. This work lies at the intersection of statistics, uncertainty quantification, high performance computing, and computational applied mathematics.

D-optimal joint best linear unbiased prediction of order statistics

On simultaneous best linear unbiased prediction of future order statistics and associated properties

A Two-Step PLS Inspired Method for Linear Prediction with Group Effect

Electronic structure study by means of X-ray spectroscopy and theoretical calculations of the "ferric star" single molecule magnet

Spatial best linear unbiased prediction: a computational mathematics approach for high dimensional massive datasets

A New Technique of Intelligent Constructing Unbiased Prediction Limits on Future Order Statistics Coming from an Inverse Gaussian Distribution under Parametric Uncertainty

Optimizing dynamic predictions from joint models using super learning

Delayed kernels for longitudinal survival analysis and dynamic prediction

Joint modeling of zero-inflated longitudinal measurements and time-to-event outcomes with applications to dynamic prediction

Combining Dynamic Predictions From Joint Models for Longitudinal and Time-to-Event Data Using Bayesian Model Averaging

Selecting Optimal Decisions Via Distributionally Robust Nearest-Neighbor Regression.

Dynamic prediction of survival using multivariate functional principal component analysis: A strict landmarking approach

A Statistical Perspective on Linear Programs with Uncertain Parameters

Adapting Best Linear Unbiased Prediction (BLUP) for Timely Genetic Evaluation: I. Progeny Traits in a Single Contemporary Group for Each Sex

Optimal Bayesian fault prediction scheme for a partially observable system subject to random failure

Nearly minimax empirical Bayesian prediction of independent Poisson observables

Best linear unbiased prediction for linear combinations in general mixed linear models

Joint reconstruction and prediction regions for unobserved exponential lifetimes under doubly Type II censoring

Bootstrap prediction inference of nonlinear autoregressive models