Abstract:BACKGROUND: In 1967, Frederick Lord posed a conundrum that has confused scientists for over 50-years. Subsequently named Lord's 'paradox', the puzzle centres on the observation that two common approach to analyses of 'change' between two time-points can produce radically different results. Approach 1 involves analysing the follow-up minus baseline (i.e., 'change score') and Approach 2 involves analysing the follow-up conditional on baseline. METHODS: At the heart of Lord's 'paradox' lies another puzzle concerning the use of 'change scores' in observational data. Using directed acyclic graphs and data simulations, we introduce, explore, and explain the 'paradox', consider the philosophy of change, and discuss the warnings and lessons of this 50-year puzzle. RESULTS: Understanding Lord's 'paradox' starts with recognising that a variable may change for three reasons: (A) 'endogenous change', which represents simple changes in scale, (B) 'random change', which represents change due to random processes, and (C) 'exogenous change', which represents all non-endogenous, non-random change. Unfortunately, in observational data, neither Approach 1 nor Approach 2 are able to reliably estimate the causes of 'exogenous change'. Approach 1 evaluates obscure estimands with little, if any, real-world interpretation. Approach 2 is susceptible to mediator-outcome confounding and cannot distinguish exogenous change from random change. Valid and precise estimates of a useful causal estimand instead require appropriate multivariate methods (such as g-methods) and more than two measures of the outcome. CONCLUSION: Lord's 'paradox' reiterates the dangers of analysing change scores in observational data and highlights the importance of considering causal questions within a causal framework.

An Empirical Unravelling of Lord's Paradox

Lord's 'paradox' explained: the 50-year warning on the use of 'change scores' in observational data

Research Designs and Causal Inferences: on Lord’s Paradox

ON LORD'S PARADOX

The covariate-adjusted residual estimator and its use in both randomized trials and observational settings

Real Effect or Bias? Good Practices for Evaluating the Robustness of Evidence From Comparative Observational Studies Through Quantitative Sensitivity Analysis for Unmeasured Confounding

Recanting witness and natural direct effects: Violations of assumptions or definitions?

A Monte Carlo Study of the Effects of Common Method Variance on Significance Testing and Parameter Bias in Hierarchical Linear Modeling

Lurking Inferential Monsters? Quantifying bias in non-experimental evaluations of school programs

Identification and estimation of causal effects in the presence of confounded principal strata

Several untestable assumptions in epidemiological observational studies: control models and mechanism of missing data

Evaluating the Health Impact of Large-Scale Public Policy Changes: Classical and Novel Approaches

On the reliability of published findings using the regression discontinuity design in political science

Comparing Targeting Strategies for Maximizing Social Welfare with Limited Resources

Identifying and estimating effects of sustained interventions under parallel trends assumptions

Sharpening bounds on principal effects with covariates

Are Difference-in-Differences Estimates Blowing Smoke? A Cautionary Tale From an Oft-Used Natural Experiment

Ecological Fallacy and Covariates: New Insights based on Multilevel Modelling of Individual Data

Challenges in interpreting results from ‘multiple regression’ when there is interaction between covariates

Weighted Approach for Estimating Effects in Principal Strata with Missing Data for a Categorical Post-Baseline Variable in Randomized Controlled Trials

Managing the exponential growth of mendelian randomization studies