Abstract:The critical importance of justifying the inclusion of covariates is a facet often overlooked in data analysis. While the incorporation of covariates typically follows informal guidelines, we argue for a comprehensive exploration of underlying principles to avoid significant statistical and interpretational challenges. Our focus is on addressing three common yet problematic practices: the indiscriminate lumping of covariates, the lack of rationale for covariate inclusion, and the oversight of potential issues in result reporting. These challenges, prevalent in neuroimaging models involving covariates such as reaction time, demographics, and morphometric measures, can introduce biases, including overestimation, underestimation, masking, sign flipping, or spurious effects. Our exploration of causal inference principles underscores the pivotal role of domain knowledge in guiding covariate selection, challenging the common reliance on statistical measures. This understanding carries implications for experimental design, model-building, and result interpretation. We draw connections between these insights and reproducibility concerns, specifically addressing the selection bias resulting from the widespread practice of strict thresholding, akin to the logical pitfall associated with "double dipping." Recommendations for robust data analysis involving covariates encompass explicit research question statements, justified covariate inclusions/exclusions, centering quantitative variables for interpretability, appropriate reporting of effect estimates, and advocating a "highlight, don't hide" approach in result reporting. These suggestions are intended to enhance the robustness, transparency, and reproducibility of covariate-driven analyses, encompassing investigations involving consortium datasets such as ABCD and UK Biobank. We discuss how researchers can use a transparent depiction of the covariate relationships to enhance the ethos of open science and promote research reproducibility.

That’s a Lot to Process! Pitfalls of Popular Path Models

Causal inference for psychologists who think that causal inference is not for them

Causal Models for Mediation Analysis: an Introduction to Structural Mean Models.

Counterfactual Graphical Models for Longitudinal Mediation Analysis With Unobserved Confounding

How to define and test an indirect moderation model: The missing link in regression-based path models

Dos and don'ts for conducting mediation analysis: A commentary with practical tips to avoid common problems.

Enhancing inferences and conclusions in body image focused non-experimental research via a causal modelling approach: A tutorial

Extending a rational process model of causal reasoning: Assessing Markov violations and explaining away with inhibitory causal relations.

Clarifying causal mediation analysis for the applied researcher: Defining effects based on what we want to learn

Through the lens of causal inference: Decisions and pitfalls of covariate selection

Statistical Approaches for Enhancing Causal Interpretation of the M to Y Relation in Mediation Analysis

The Failings of Conventional Mediation Analysis and a Design-Based Alternative

The Causal Roadmap and Simulations to Improve the Rigor and Reproducibility of Real-data Applications

Time and Other Considerations in Mediation Design

Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference

More powerful parameter tests? No, rather biased parameter estimates. Some reflections on path analysis with weighted composites

Misleading Heuristics and Moderated Multiple Regression Models

Beyond a tandem analysis of SEM and PROCESS: Use of PLS-SEM for mediation analyses!

Practical challenges in mediation analysis: a guide for applied researchers

Credible causal inference beyond toy models

Forward stability and model path selection