Abstract:Debiasing is a fundamental concept in high-dimensional statistics. While degrees-of-freedom adjustment is the state-of-the-art technique in high-dimensional linear regression, it is limited to i.i.d. samples and sub-Gaussian covariates. These constraints hinder its broader practical use. Here, we introduce Spectrum-Aware Debiasing--a novel method for high-dimensional regression. Our approach applies to problems with structured dependencies, heavy tails, and low-rank structures. Our method achieves debiasing through a rescaled gradient descent step, deriving the rescaling factor using spectral information of the sample covariance matrix. The spectrum-based approach enables accurate debiasing in much broader contexts. We study the common modern regime where the number of features and samples scale proportionally. We establish asymptotic normality of our proposed estimator (suitably centered and scaled) under various convergence notions when the covariates are right-rotationally invariant. Such designs have garnered recent attention due to their crucial role in compressed sensing. Furthermore, we devise a consistent estimator for its asymptotic variance. Our work has two notable by-products: first, we use Spectrum-Aware Debiasing to correct bias in principal components regression (PCR), providing the first debiased PCR estimator in high dimensions. Second, we introduce a principled test for checking alignment between the signal and the eigenvectors of the sample covariance matrix. This test is independently valuable for statistical methods developed using approximate message passing, leave-one-out, or convex Gaussian min-max theorems. We demonstrate our method through simulated and real data experiments. Technically, we connect approximate message passing algorithms with debiasing and provide the first proof of the Cauchy property of vector approximate message passing (V-AMP).

Bispectrum Unbiasing for Dilation-Invariant Multi-reference Alignment

Bispectrum Unbiasing for Dilation-Invariant Multi-Reference Alignment

Sparse Multi-Reference Alignment : Phase Retrieval, Uniform Uncertainty Principles and the Beltway Problem

A Spectral Method for Stable Bispectrum Inversion with Application to Multireference Alignment

Rates of estimation for high-dimensional multi-reference alignment

Minimax-optimal estimation for sparse multi-reference alignment with collision-free signals

The generalized method of moments for multi-reference alignment

The sample complexity of sparse multi-reference alignment and single-particle cryo-electron microscopy

Denoiser-based projections for 2-D super-resolution multi-reference alignment

Harmonic Alignment

Concurrent bias correction in hemodynamic data assimilation.

Joint reconstruction and bias field correction for undersampled MR imaging

DMRA: An Adaptive Line Spectrum Estimation Method through Dynamical Multi-Resolution of Atoms

Sequence adaptive field-imperfection estimation (SAFE): retrospective estimation and correction of $B_1^+$ and $B_0$ inhomogeneities for enhanced MRF quantification

Bayesian Perspective for Orientation Estimation in Cryo-EM and Cryo-ET

Intensity warping for multisite MRI harmonization

Rotationally invariant image representation for viewing direction classification in cryo-EM

Spectrum-Aware Debiasing: A Modern Inference Framework with Applications to Principal Components Regression

Intrinsic alignment from multiple shear estimates: A first application to data and forecasts for Stage IV

Multi-Irreducible Spectral Synchronization for Robust Rotation Averaging

Information-based Disentangled Representation Learning for Unsupervised MR Harmonization