Abstract:Objectives: To examine the impact of patient-mix on National Health Service (NHS) acute hospital trust scores in two national NHS patient surveys. Methods: Secondary analysis of 2012 patient survey data for 57,915 adult inpatients at 142 NHS acute hospital trusts and 45,263 adult emergency department attendees at 146 NHS acute hospital trusts in England. Changes in trust scores for selected questions, ranks, inter-trust variance and score-based performance bands were examined using three methods: no adjustment for case-mix; the current standardization method with weighting for age, sex and, for inpatients only, admission method; and a regression model adjusting in addition for ethnicity, presence of a long-term condition, proxy response (inpatients only) and previous emergency attendances (emergency department survey only). Results: For both surveys, all the variables examined were associated with patients' responses and affected inter-trust variance in scores, although the direction and strength of impact differed between variables. Inter-trust variance was generally greatest for the unadjusted scores and lowest for scores derived from the full regression model. Although trust scores derived from the three methods were highly correlated (Kendall's tau coefficients 0.70-0.94), up to 14% of trusts had discordant ranks of when the standardization and regression methods were compared. Depending on the survey and question, up to 14 trusts changed performance bands when the regression model with its fuller case-mix adjustment was used rather than the current standardization method. Conclusions: More comprehensive case-mix adjustment of patient survey data than the current limited adjustment reduces performance variation between NHS acute hospital trusts and alters the comparative performance bands of some trusts. Given the use of these data for high-impact purposes such as performance assessment, regulation, commissioning, quality improvement and patient choice, a review of the long-standing method for analysing patient survey data would be timely, and could improve rigour and comparability across the NHS. Performance comparisons need to be perceived as fair and scientifically robust to maintain confidence in publicly reported data, and to support their use by both the public and the NHS.

Hospital quality risk standardization via approximate balancing weights

Performance Evaluation of Inpatient Service in Beijing: a Horizontal Comparison with Risk Adjustment Based on Diagnosis Related Groups

Scalable kernel balancing weights in a nationwide observational study of hospital profit status and heart attack outcomes

Optimization‐Based Stable Balancing Weights Versus Propensity Score Weighting for Samples With High Covariate Imbalance

Balancing versus modelling in weighted analysis of non‐randomised studies with survival outcomes: A simulation study

Separating the wheat from the chaff: How to measure hospital quality in routine data?

Patient Safety in Hospitals - a Bayesian Analysis of Unobservable Hospital and Specialty Level Risk Factors.

Mortality Rate Estimation and Standardization for Public Reporting: Medicare's Hospital Compare

Estimating Risk-Adjusted Hospital Performance

Using Balancing Weights to Target the Treatment Effect on the Treated when Overlap is Poor

IDENTIFYING SOURCES OF INEFFICIENCY IN HEALTHCARE

Impact of case-mix on comparisons of patient-reported experience in NHS acute hospital trusts in England

Estimating Racial Disparities in Emergency General Surgery

The impact of surgical volume on hospital ranking using the standardized infection ratio

Accounting for total variation and robustness in profiling health care providers

A Case Study of Stochastic Optimization in Health Policy: Problem Formulation and Preliminary Results

Online Profile Monitoring for Surgical Outcomes Using a Weighted Score Test

Split‐sample reliability estimation in health care quality measurement: Once is not enough

Hospital-Wide Inpatient Flow Optimization

Survey calibration for causal inference: a simple method to balance covariate distributions

Comparing analytical strategies for balancing site-level characteristics in stepped-wedge cluster randomized trials: a simulation study