Abstract:The critical importance of justifying the inclusion of covariates is a facet often overlooked in data analysis. While the incorporation of covariates typically follows informal guidelines, we argue for a comprehensive exploration of underlying principles to avoid significant statistical and interpretational challenges. Our focus is on addressing three common yet problematic practices: the indiscriminate lumping of covariates, the lack of rationale for covariate inclusion, and the oversight of potential issues in result reporting. These challenges, prevalent in neuroimaging models involving covariates such as reaction time, demographics, and morphometric measures, can introduce biases, including overestimation, underestimation, masking, sign flipping, or spurious effects. Our exploration of causal inference principles underscores the pivotal role of domain knowledge in guiding covariate selection, challenging the common reliance on statistical measures. This understanding carries implications for experimental design, model-building, and result interpretation. We draw connections between these insights and reproducibility concerns, specifically addressing the selection bias resulting from the widespread practice of strict thresholding, akin to the logical pitfall associated with "double dipping." Recommendations for robust data analysis involving covariates encompass explicit research question statements, justified covariate inclusions/exclusions, centering quantitative variables for interpretability, appropriate reporting of effect estimates, and advocating a "highlight, don't hide" approach in result reporting. These suggestions are intended to enhance the robustness, transparency, and reproducibility of covariate-driven analyses, encompassing investigations involving consortium datasets such as ABCD and UK Biobank. We discuss how researchers can use a transparent depiction of the covariate relationships to enhance the ethos of open science and promote research reproducibility.

The causal interpretation of estimated associations in regression models

Causal Interpretation of Estimands Defined by Exposure Mappings

What can be estimated? Identifiability, estimability, causal inference and ill-posed inverse problems

Estimating complex causal effects from incomplete observational data

Causal inference under over-simplified longitudinal causal models

Identifying Total Causal Effects in Linear Models under Partial Homoscedasticity

On Estimating Regression-Based Causal Effects Using Sufficient Dimension Reduction

Identifiability, Stratification and Minimum Variance Estimation of Causal Effects

Through the lens of causal inference: Decisions and pitfalls of covariate selection

On the Parameter Identifiability of Partially Observed Linear Causal Models

Causal Interpretation of Regressions With Ranks

A Causal Framework for Observational Studies of Discrimination

Nonparametric identification and efficient estimation of causal effects with instrumental variables

A cautious approach to constraint-based causal model selection

Causal inference using regression-based statistical control: Confusion in Econometrics

Objective causal predictions from observational data

Linking a predictive model to causal effect estimation

Causal Inference with High-dimensional Discrete Covariates

On the implied weights of linear regression for causal inference

Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference

Estimating Possible Causal Effects with Latent Variables Via Adjustment.