Abstract:The critical importance of justifying the inclusion of covariates is a facet often overlooked in data analysis. While the incorporation of covariates typically follows informal guidelines, we argue for a comprehensive exploration of underlying principles to avoid significant statistical and interpretational challenges. Our focus is on addressing three common yet problematic practices: the indiscriminate lumping of covariates, the lack of rationale for covariate inclusion, and the oversight of potential issues in result reporting. These challenges, prevalent in neuroimaging models involving covariates such as reaction time, demographics, and morphometric measures, can introduce biases, including overestimation, underestimation, masking, sign flipping, or spurious effects. Our exploration of causal inference principles underscores the pivotal role of domain knowledge in guiding covariate selection, challenging the common reliance on statistical measures. This understanding carries implications for experimental design, model-building, and result interpretation. We draw connections between these insights and reproducibility concerns, specifically addressing the selection bias resulting from the widespread practice of strict thresholding, akin to the logical pitfall associated with "double dipping." Recommendations for robust data analysis involving covariates encompass explicit research question statements, justified covariate inclusions/exclusions, centering quantitative variables for interpretability, appropriate reporting of effect estimates, and advocating a "highlight, don't hide" approach in result reporting. These suggestions are intended to enhance the robustness, transparency, and reproducibility of covariate-driven analyses, encompassing investigations involving consortium datasets such as ABCD and UK Biobank. We discuss how researchers can use a transparent depiction of the covariate relationships to enhance the ethos of open science and promote research reproducibility.

Through the lens of causal inference: Decisions and pitfalls of covariate selection

Control without cause: How covariate control biases our insights into brain architecture and pathology

Causal Inference and Counterfactual Prediction in Machine Learning for Actionable Healthcare

Learning sources of variability from high-dimensional observational studies

An Evaluation Toolkit to Guide Model Selection and Cohort Definition in Causal Inference

Causal inference for psychologists who think that causal inference is not for them

Methodological Challenges in Causal Research on Racial and Ethnic Patterns of Cognitive Trajectories: Measurement, Selection, and Bias

A Comparative Study of Design-Based and Analysis-Based Approaches to Causal Inference with Observational Data

A robust and efficient approach to causal inference based on sparse sufficient dimension reduction

The Causal Roadmap and Simulations to Improve the Rigor and Reproducibility of Real-data Applications

Estimating Causal Effects With Partial Covariates For Clinical Interpretability

Misstatements, misperceptions, and mistakes in controlling for covariates in observational research

Causal Inference with Latent Variables: Recent Advances and Future Prospectives

Propensity Score Adapted Covariate Selection for Causal Inference

Local Learning for Covariate Selection in Nonparametric Causal Effect Estimation with Latent Variables

Causal inference on human behaviour

Using causal inference to avoid fallouts in data-driven parametric analysis: a case study in the architecture, engineering, and construction industry

Causal inference with Machine Learning-Based Covariate Representation

Enhancing causal inference in population-based neuroimaging data in children and adolescents

Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference

Causal Inference with High-dimensional Discrete Covariates