Abstract:In this paper we study the asymptotics of linear regression in settings where the covariates exhibit a linear dependency structure, departing from the standard assumption of independence. We model the covariates using stochastic processes with spatio-temporal covariance and analyze the performance of ridge regression in the high-dimensional proportional regime, where the number of samples and feature dimensions grow proportionally. A Gaussian universality theorem is proven, demonstrating that the asymptotics are invariant under replacing the covariates with Gaussian vectors preserving mean and covariance. Next, leveraging tools from random matrix theory, we derive precise characterizations of the estimation error. The estimation error is characterized by a fixed-point equation involving the spectral properties of the spatio-temporal covariance matrices, enabling efficient computation. We then study optimal regularization, overparameterization, and the double descent phenomenon in the context of dependent data. Simulations validate our theoretical predictions, shedding light on how dependencies influence estimation error and the choice of regularization parameters.

Asymptotics of Estimates in Constrained Nonlinear Regression with Long-Range Dependent Innovations

Asymptotic Behavior for S-estimators in Random Design Linear Model with Long-Range-dependent Errors

Asymptotic Distributions of Innovation Density Estimators in Linear Processes

ROBUST ESTIMATION IN PARAMETRIC TIME SERIES MODELS UNDER LONG‐ AND SHORT‐RANGE‐DEPENDENT STRUCTURES

Asymptotic expansion for nonparametric M-estimator in a nonlinear regression model with long-memory errors

Asymptotic Properties of the Least Squares Estimator in Local to Unity Processes with Fractional Gaussian Noise

Regularization and variable selection for infinite variance autoregressive models

Mildly Explosive Autoregression with Anti-persistent Errors

Adaptive Lasso in high-dimensional settings

Asymptotic Theory for Lad Estimation of Moderate Deviations from A Unit Root

Asymptotic Theory of Least Distances Estimate in Multivariate Linear Models

Asymptotic of the Lr-norm of density estimators in the autoregressive time series

Asymptotics of Estimators for Nonparametric Multivariate Regression Models with Long Memory

Asymptotics of Linear Regression with Linearly Dependent Data

Asymptotics for A Class of Dependent Random Variables

Change in Non-Parametric Regression with Long Memory Errors

Inference for High-Dimensional Linear Expectile Regression with De-Biasing Method

EfficientL1estimation and Related Inferences in Linear Regression with Unknown Form of Heteroscedasticity

On the Asymptotics of Marginal Regression Splines with Longitudinal Data

Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumption

Asymptotics of nonparametric L-1 regression models with dependent data