Abstract:The critical importance of justifying the inclusion of covariates is a facet often overlooked in data analysis. While the incorporation of covariates typically follows informal guidelines, we argue for a comprehensive exploration of underlying principles to avoid significant statistical and interpretational challenges. Our focus is on addressing three common yet problematic practices: the indiscriminate lumping of covariates, the lack of rationale for covariate inclusion, and the oversight of potential issues in result reporting. These challenges, prevalent in neuroimaging models involving covariates such as reaction time, demographics, and morphometric measures, can introduce biases, including overestimation, underestimation, masking, sign flipping, or spurious effects. Our exploration of causal inference principles underscores the pivotal role of domain knowledge in guiding covariate selection, challenging the common reliance on statistical measures. This understanding carries implications for experimental design, model-building, and result interpretation. We draw connections between these insights and reproducibility concerns, specifically addressing the selection bias resulting from the widespread practice of strict thresholding, akin to the logical pitfall associated with "double dipping." Recommendations for robust data analysis involving covariates encompass explicit research question statements, justified covariate inclusions/exclusions, centering quantitative variables for interpretability, appropriate reporting of effect estimates, and advocating a "highlight, don't hide" approach in result reporting. These suggestions are intended to enhance the robustness, transparency, and reproducibility of covariate-driven analyses, encompassing investigations involving consortium datasets such as ABCD and UK Biobank. We discuss how researchers can use a transparent depiction of the covariate relationships to enhance the ethos of open science and promote research reproducibility.

Challenges in interpreting results from ‘multiple regression’ when there is interaction between covariates

Interpreting moderated multiple regression: A comment on Van Iddekinge, Aguinis, Mackey, and DeOrtentiis (2018).

Regression analyses and their particularities in observational studies

The covariate-adjusted residual estimator and its use in both randomized trials and observational settings

DEALING WITH MULTIVARIATE OUTCOMES IN STUDIES FOR CAUSAL EFFECTS

The impact of neglected confounding and interactions in mixed-effects meta-regression

The Use of Covariate Adjustment in Randomized Controlled Trials: An Overview

Interaction tests with covariate-adaptive randomization

Asymmetric Independence Model for Detecting Interactions between Variables

Mediation and moderation of treatment effects in randomised controlled trials of complex interventions

Regression-based multiple treatment effect estimation under covariate-adaptive randomization

Covariate adjustment and prediction of mean response in randomised trials

Through the lens of causal inference: Decisions and pitfalls of covariate selection

Conditional and Unconditional Treatment Effects in Randomized Clinical Trials: Estimands, Estimation, and Interpretation

Implications of confounding from unmodeled interactions between explanatory variables when using latent variable regression models to make inferences

Estimands and Cumulative Incidence Function Regression in Clinical Trials: Some New Results on Interpretability and Robustness

Regression Analysis for Covariate-Adaptive Randomization: A Robust and Efficient Inference Perspective

Interpretation and Potential Biases of Mendelian Randomization Estimates With Time-Varying Exposures.

Choosing statistical models to assess biological interaction as a departure from additivity of effects

Subtleties in the interpretation of hazard ratios

A Comparison of Approaches to Control for Confounding Factors by Regression Models