Abstract:There are many scenarios such as the electronic health records where the outcome is much more difficult to collect than the covariates. In this paper, we consider the linear regression problem with such a data structure under the high dimensionality. Our goal is to investigate when and how the unlabeled data can be exploited to improve the estimation and inference of the regression parameters in linear models, especially in light of the fact that such linear models may be misspecified in data analysis. In particular, we address the following two important questions. (1) Can we use the labeled data as well as the unlabeled data to construct a semi-supervised estimator such that its convergence rate is faster than the supervised estimators? (2) Can we construct confidence intervals or hypothesis tests that are guaranteed to be more efficient or powerful than the supervised estimators? To address the first question, we establish the minimax lower bound for parameter estimation in the semi-supervised setting. We show that the upper bound from the supervised estimators that only use the labeled data cannot attain this lower bound. We close this gap by proposing a new semi-supervised estimator which attains the lower bound. To address the second question, based on our proposed semi-supervised estimator, we propose two additional estimators for semi-supervised inference, the efficient estimator and the safe estimator. The former is fully efficient if the unknown conditional mean function is estimated consistently, but may not be more efficient than the supervised approach otherwise. The latter usually does not aim to provide fully efficient inference, but is guaranteed to be no worse than the supervised approach, no matter whether the linear model is correctly specified or the conditional mean function is consistently estimated.

Statistical Analysis of a Linear Regression Model with Restrictions and Superfluous Variables

Statistical Inference on Restricted Partially Linear Additive Errors-in-variables Models

Statistical Inference of Seemingly Unrelated Partially Linear Additive Regression Models

Statistical inference for semiparametric varying-coefficient partially linear models with error-prone linear covariates

On the Stochastic Restricted Liu Estimator under Misspecification due to Inclusion of Some Superfluous Variables

On the Restricted Almost Unbiased Estimators in Linear Regression

Restricted Estimation in Partially Linear Errors-In-Variables Models

Statistical inference in partially linear varying-coefficient models with missing responses at random

Equivalence of Two Tests in Varying Coefficient Partially Linear Errors in Variable Model with Missing Responses

On the Stochastic Restricted Almost Unbiased Estimators in Linear Regression Model

Models as Approximations I: Consequences Illustrated with Linear Regression

Statistical Inference on Partially Linear Additive Models with Missing Response Variables and Error-Prone Covariates

Preliminary test almost unbiased ridge estimator in a linear regression model with multivariate Student-t errors

Asymptotics of Linear Regression with Linearly Dependent Data

A new analysis of optimal estimation and prediction under linear mixed models

Selection of Regression Models under Linear Restrictions for Fixed and Random Designs

Some Overall Properties of Seemingly Unrelated Regression Models

A Stochastic Restricted Ridge Regression Estimator in Partially Linear Varying Coefficient Models

Penalized Profile Least Squares-Based Statistical Inference for Varying Coefficient Partially Linear Errors-in-variables Models

Optimal Semi-supervised Estimation and Inference for High-dimensional Linear Regression

A robust and efficient estimation and variable selection method for partially linear models with large-dimensional covariates