Lord's 'paradox' explained: the 50-year warning on the use of 'change scores' in observational data
Peter W.G. Tennant,Georgia D. Tomova,Eleanor J. Murray,Kellyn F. Arnold,Matthew P. Fox,Mark S. Gilthorpe
DOI: https://doi.org/10.48550/arXiv.2302.01822
2023-02-03
Abstract:BACKGROUND: In 1967, Frederick Lord posed a conundrum that has confused scientists for over 50-years. Subsequently named Lord's 'paradox', the puzzle centres on the observation that two common approach to analyses of 'change' between two time-points can produce radically different results. Approach 1 involves analysing the follow-up minus baseline (i.e., 'change score') and Approach 2 involves analysing the follow-up conditional on baseline. METHODS: At the heart of Lord's 'paradox' lies another puzzle concerning the use of 'change scores' in observational data. Using directed acyclic graphs and data simulations, we introduce, explore, and explain the 'paradox', consider the philosophy of change, and discuss the warnings and lessons of this 50-year puzzle. RESULTS: Understanding Lord's 'paradox' starts with recognising that a variable may change for three reasons: (A) 'endogenous change', which represents simple changes in scale, (B) 'random change', which represents change due to random processes, and (C) 'exogenous change', which represents all non-endogenous, non-random change. Unfortunately, in observational data, neither Approach 1 nor Approach 2 are able to reliably estimate the causes of 'exogenous change'. Approach 1 evaluates obscure estimands with little, if any, real-world interpretation. Approach 2 is susceptible to mediator-outcome confounding and cannot distinguish exogenous change from random change. Valid and precise estimates of a useful causal estimand instead require appropriate multivariate methods (such as g-methods) and more than two measures of the outcome. CONCLUSION: Lord's 'paradox' reiterates the dangers of analysing change scores in observational data and highlights the importance of considering causal questions within a causal framework.
Methodology