Abstract:It remains unclear when standard systematic reviews and meta-analyses that rely on published aggregate data (AD) can provide robust clinical conclusions. We aimed to compare the results from a large cohort of systematic reviews and meta-analyses based on individual participant data (IPD) with meta-analyses of published AD, to establish when the latter are most likely to be reliable and when the IPD approach might be required.We used 18 cancer systematic reviews that included IPD meta-analyses: all of those completed and published by the Meta-analysis Group of the MRC Clinical Trials Unit from 1991 to 2010. We extracted or estimated hazard ratios (HRs) and standard errors (SEs) for survival from trial reports and compared these with IPD equivalents at both the trial and meta-analysis level. We also extracted or estimated the number of events. We used paired t tests to assess whether HRs and SEs from published AD differed on average from those from IPD. We assessed agreement, and whether this was associated with trial or meta-analysis characteristics, using the approach of Bland and Altman. The 18 systematic reviews comprised 238 unique trials or trial comparisons, including 37,082 participants. A HR and SE could be generated for 127 trials, representing 53% of the trials and approximately 79% of eligible participants. On average, trial HRs derived from published AD were slightly more in favour of the research interventions than those from IPD (HRAD to HRIPD ratio = 0.95, p = 0.007), but the limits of agreement show that for individual trials, the HRs could deviate substantially. These limits narrowed with an increasing number of participants (p < 0.001) or a greater number (p < 0.001) or proportion (p < 0.001) of events in the AD. On average, meta-analysis HRs from published AD slightly tended to favour the research interventions whether based on fixed-effect (HRAD to HRIPD ratio = 0.97, p = 0.088) or random-effects (HRAD to HRIPD ratio = 0.96, p = 0.044) models, but the limits of agreement show that for individual meta-analyses, agreement was much more variable. These limits tended to narrow with an increasing number (p = 0.077) or proportion of events (p = 0.11) in the AD. However, even when the information size of the AD was large, individual meta-analysis HRs could still differ from their IPD equivalents by a relative 10% in favour of the research intervention to 5% in favour of control. We utilised the results to construct a decision tree for assessing whether an AD meta-analysis includes sufficient information, and when estimates of effects are most likely to be reliable. A lack of power at the meta-analysis level may have prevented us identifying additional factors associated with the reliability of AD meta-analyses, and we cannot be sure that our results are generalisable to all outcomes and effect measures.In this study we found that HRs from published AD were most likely to agree with those from IPD when the information size was large. Based on these findings, we provide guidance for determining systematically when standard AD meta-analysis will likely generate robust clinical conclusions, and when the IPD approach will add considerable value.

Efficient Integration of Aggregate Data and Individual Patient Data in One-Way Mixed Models

Meta‐analysis of a Continuous Outcome Combining Individual Patient Data and Aggregate Data: a Method Based on Simulated Individual Patient Data

Meta-analysis of continuous outcomes combining individual patient data and aggregate data

Combining Survival Trials Using Aggregate Data Based on Misspecified Models

A propensity score weighting approach to integrate aggregated data in random-effect individual-level data meta-analysis

Random-effects Meta-Analysis of Combined Outcomes Based on Reconstructions of Individual Patient Data

A novel approach for identifying and addressing case-mix heterogeneity in individual participant data meta-analysis

Integrative analysis of individual-level data and high-dimensional summary statistics

Individual Participant Data Meta‐analysis to Examine Interactions Between Treatment Effect and Participant‐level Covariates: Statistical Recommendations for Conduct and Planning

Combining randomized and non‐randomized data to predict heterogeneous effects of competing treatments

Analyzing Treatment Effect by Integrating Existing Propensity Score and Outcome Regressions with Heterogeneous Covariate Sets

Combining Primary Cohort Data with External Aggregate Information Without Assuming Comparability

Relative Efficiency Of Using Summary Versus Individual Data In Random-Effects Meta-Analysis

Efficient Algorithms for Covariate Analysis with Dynamic Data Using Nonlinear Mixed-Effects Model.

Adaptive Combination of Randomized and Observational Data

Comparison of aggregate and individual participant data approaches to meta-analysis of randomised trials: An observational study

Penalized Poisson model for network meta-analysis of individual patient time-to-event data

Extension of individual model averaging assessments to unbalanced designs and dose-response

Estimating individualized treatment effects using an individual participant data meta-analysis

On the Comparative Analysis of Average Treatment Effects Estimation Via Data Combination

Individual participant data meta-analysis to examine linear or non-linear treatment-covariate interactions at multiple time-points for a continuous outcome