Abstract:We investigate high-dimensional nonconvex penalized regression, where the number of covariates may grow at an exponential rate. Although recent asymptotic theory established that there exists a local minimum possessing the oracle property under general conditions, it is still largely an open problem how to identify the oracle estimator among potentially multiple local minima. There are two main obstacles: (1) due to the presence of multiple minima, the solution path is nonunique and is not guaranteed to contain the oracle estimator; (2) even if a solution path is known to contain the oracle estimator, the optimal tuning parameter depends on many unknown factors and is hard to estimate. To address these two challenging issues, we first prove that an easy-to-calculate calibrated CCCP algorithm produces a consistent solution path which contains the oracle estimator with probability approaching one. Furthermore, we propose a high-dimensional BIC criterion and show that it can be applied to the solution path to select the optimal tuning parameter which asymptotically identifies the oracle estimator. The theory for a general class of nonconvex penalties in the ultra-high dimensional setup is established when the random errors follow the sub-Gaussian distribution. Monte Carlo studies confirm that the calibrated CCCP algorithm combined with the proposed high-dimensional BIC has desirable performance in identifying the underlying sparsity pattern for high-dimensional data analysis.

Multivariate Calibrations with Auxiliary Information

On Multivariate Calibration Problems

Two-Level Independent Component Regression Model for Multivariate Spectroscopic Calibration

When is Multicalibration Post-Processing Necessary?

MULTIVARIATE VARYING COEFFICIENT MODEL FOR FUNCTIONAL RESPONSES

Calibrated Multivariate Regression with Application to Neural Semantic Basis Discovery

Robust Multivariate Calibration Algorithm Based on Least Median of Squares and Sequential Number Theory Optimization Method

Bridging Multicalibration and Out-of-distribution Generalization Beyond Covariate Shift

Debiased calibration estimation using generalized entropy in survey sampling

Parallel Calibration Revisited: the Second Direction for Shrinkage Estimation of Regression Coefficients Can Be As Natural and Necessary As the Traditional One.

Updating Multivariate Calibrations with the Delaunay Triangulation Method

A strategy that iteratively retains informative variables for selecting optimal variable subset in multivariate calibration.

Constrained Maximum Likelihood Estimation for Model Calibration Using Summary-Level Information From External Big Data Sources

An Improved Ensemble Method for Completely Automatic Optimization of Spectral Interval Selection in Multivariate Calibration

A Perspective Demonstration on the Importance of Variable Selection in Inverse Calibration for Complex Analytical Systems

Using Model-Assisted Calibration Methods to Improve Efficiency of Regression Analyses with Two-Phase Samples under Complex Survey Designs

The Information Domain Confidence Intervals in Univariate Linear Calibration.

Calibrating nonconvex penalized regression in ultra-high dimension

Optimization for Calibration of Survey Weights under a Large Number of Conflicting Constraints

Multivariate Calibration of Near-Infrared Spectra by Using Influential Variables

Multivariate Regression Shrinkage and Selection by Canonical Correlation Analysis