Abstract:There are many scenarios such as the electronic health records where the outcome is much more difficult to collect than the covariates. In this paper, we consider the linear regression problem with such a data structure under the high dimensionality. Our goal is to investigate when and how the unlabeled data can be exploited to improve the estimation and inference of the regression parameters in linear models, especially in light of the fact that such linear models may be misspecified in data analysis. In particular, we address the following two important questions. (1) Can we use the labeled data as well as the unlabeled data to construct a semi-supervised estimator such that its convergence rate is faster than the supervised estimators? (2) Can we construct confidence intervals or hypothesis tests that are guaranteed to be more efficient or powerful than the supervised estimators? To address the first question, we establish the minimax lower bound for parameter estimation in the semi-supervised setting. We show that the upper bound from the supervised estimators that only use the labeled data cannot attain this lower bound. We close this gap by proposing a new semi-supervised estimator which attains the lower bound. To address the second question, based on our proposed semi-supervised estimator, we propose two additional estimators for semi-supervised inference, the efficient estimator and the safe estimator. The former is fully efficient if the unknown conditional mean function is estimated consistently, but may not be more efficient than the supervised approach otherwise. The latter usually does not aim to provide fully efficient inference, but is guaranteed to be no worse than the supervised approach, no matter whether the linear model is correctly specified or the conditional mean function is consistently estimated.

Optimal and Superoptimal Convergence Rate of the Local Linear Estimator of Nonparametric Regression Function in Continuous Time

Local Linear M‐estimation in Non‐parametric Spatial Regression

The convergence of conjugate gradient method with nonmonotone line search

Uniform and Lp Convergences for Nonparametric Continuous Time Regressions with Semiparametric Applications

Local convergence rates of the nonparametric least squares estimator with applications to transfer learning

Convergence rate of inexact proximal point algorithms for operator with holder metric subregularity

Local linear M-estimation for spatial processes in fixed-design models

Robust estimation in a nonlinear cointegration model

Local Convergence of Inexact Methods under the Hölder Condition

Identifiability Implies Robust, Globally Exponentially Convergent On-line Parameter Estimation: Application to Model Reference Adaptive Control

Optimal Semi-supervised Estimation and Inference for High-dimensional Linear Regression

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Uniform and Lp Convergences of Nonparametric Estimation for Diffusion Models

RATES OF CONVERGENCE OF M-ESTIMATORS FOR PARTLY LINEAR-MODELS INVOLVING TIME-SERIES

Tight Nonparametric Convergence Rates for Stochastic Gradient Descent under the Noiseless Linear Model

Optimal Estimation of Derivatives in Nonparametric Regression

Strong convergence rates of several estimators in semiparametric varying-coefficient partially linear models

Estimation of Convergence Rate for Multi-Regression Learning Algorithm

The Local Linear <path id="x1D440" d="M998 650l-8 -28q-71 -4 -86 -16t-22 -69l-50 -397q-3 -28 -4.5 -44t2 -29t6.5 -18.5t17 -10.5t24.5 -6.5t37.5 -3.5l-8 -28h-271l7 28q63 6 78 22t25 90l60 415h-2l-353 -552h-23l-130 536h-2l-70 -254q-44 -158 -47 -188q-5 -38 9 -51t71 -18l-6 -28h-241l8 28

Optimal Rates for Coefficient-Based Regularized Regression

On the Convergence Rate of the Inexact Levenberg-Marquardt Method