An Introduction to Meta-Analysis.
Eric J. Lehrer,Ming Wang,Yilun Sun,Nicholas G. Zaorsky
DOI: https://doi.org/10.1016/j.ijrobp.2022.07.1831
2023-01-01
Abstract:•Meta-analysis is a powerful tool that can be used to pool multiple studies and increase statistical power.•Performing a thorough and registered systematic review before the meta-analysis is essential to ensure that all pertinent studies are included.•Effect sizes should be carefully selected with well-designed summary statistics for analysis, such as incidence proportions, hazard ratios, and odds ratios.•Pooled effect sizes obtained from a meta-analysis must be interpreted in the context of the quality of each of the included studies.•Multiple methods exist to determine the presence and quantify the impact of study heterogeneity and publication bias. A 65-year-old man presents to your clinic to discuss radiation treatment options after receiving a diagnosis of prostate cancer. Based on history, physical, and imaging, he has prostate adenocarcinoma, AJCC stage II (cT1c N0 M0) Gleason Grade Group 2 with PSA 10 ng/mL. He would like to pursue definitive treatment with external beam radiation therapy. After a detailed discussion the patient opts to proceed with hypofractionated intensity modulated radiation therapy (IMRT). During the discussion about treatment-related adverse events, the patient inquires as to what his risk for urinary toxicity would be several years after treatment. Although multiple randomized controlled trials (RCTs) have been performed comparing hypofractionated IMRT to conventionally fractionated IMRT, these trials had some differences, such as risk group of included patients, inclusion or omission of pelvic nodal irradiation, and extent of seminal vesicle coverage in the radiation treatment volume. Therefore, you decide to synthesize the individual effect sizes from each of these studies using meta-analysis to obtain a high-powered pooled effect size. A meta-analysis using study-level data is a way to address this question. However, what is the most appropriate approach in performing a study-level data meta-analysis? A meta-analysis is a statistical method that allows investigators to quantitatively combine the results of multiple studies identified in a systematic review, as shown in Fig. 1. This approach allows several potential individual study limitations to be addressed. First, individual studies may be underpowered, making it difficult to detect the presence of a treatment effect. Meta-analysis allows for the inclusion of multiple studies, thus increasing the power to detect the presence of a treatment effect, as well as its precision. Second, meta-analysis allows for evaluation of the consistency of effect sizes between individual studies addressing similar research questions. When these effect sizes are relatively consistent, the pooled effect size obtained from the meta-analysis allows for highly accurate and precise estimates that can be used for clinical decision-making. If the study effect sizes vary significantly between studies, a meta-analysis allows for the quantification and ways to explore the sources of these variabilities. Depending on the clinical question and type of outcome of interest, different effect size measures can be used, such as proportions, odds ratios, hazard ratios, risk ratios, and mean differences. Additionally, a transformation of effect measure (e.g., log-transform) may be needed due to the normality assumption for conventional meta-analytical model fitting and inference. Although multiple types of meta-analyses are used in the medical literature, the most frequently encountered are those using study-level data, which are the focus of this article. In these meta-analyses, previously published reports are collected and summary statistics from each of these studies are quantitatively combined to generate a summary effect size. Because the data being included have already been published and are not reported by individual patients, data sharing agreements and institutional review board approval are not typically required. Thus, this is a commonly used approach by many investigators. Although study-level meta-analysis is a powerful tool, the lack of individual patient data can be a significant limitation. Thus, it is essential for investigators to conduct a rigorous systematic review and carefully assess each candidate study before inclusion in the meta-analysis. Performing a comprehensive systematic review before any analyses is an essential component of any meta-analysis. Before undertaking a systematic review, a detailed protocol should be developed that outlines the literature search and selection process, data extraction, and planned analyses. The protocol may be included in a supplementary file of the article, or it can be registered online. By prospectively registering a protocol, investigators can mitigate the risk of reporting bias, which allows for higher-quality systematic reviews.1Page MJ Altman DG Shamseer L et al.Reproducible research practices are underused in systematic reviews of biomedical interventions.J Clin Epidemiol. 2018; 94: 8-18Abstract Full Text Full Text PDF PubMed Scopus (65) Google Scholar Protocol registration also reduces the chance of duplicating a study already in process. The International Prospective Register of Systematic Reviews (PROSPERO) is a widely used registry that allows any systematic review to be registered as long as it reports on a health-related outcome.2Booth A Clarke M Dooley G et al.The nuts and bolts of PROSPERO: An international prospective register of systematic reviews.Syst Rev. 2012; 1: 2Crossref PubMed Scopus (752) Google Scholar Online registration is not mandatory. The first step in performing a systematic review is designing an appropriate clinical question. A clinical question should be formulated using the Population, Intervention, Comparison, Objective, and Study (PICOS) design format. Investigators may consider consulting a medical librarian. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) provide guidelines that are aimed at improving reporting of systematic reviews and meta-analyses.3Moher D Liberati A Tetzlaff J Altman DG Group P. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement.Ann Intern Med. 2009; 151 (W64): 264-269Crossref PubMed Scopus (19438) Google Scholar The Cochrane Handbook for Systematic Reviews of Interventions is the official guide that describes the details of the systematic review process and should be used.4Higgins JP, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Review of Interventions Version 6.2 (updated February 2021). Cochrane, 2021. Available at: https://training.cochrane.org/handbook/current. Accessed 7 January, 2022Google Scholar A high-quality systematic review is an essential component of any study-level meta-analysis (Fig. 1). It is at this stage that articles are selected for inclusion; pertient datapoints are then extracted from each of these studies. The PRISMA flow diagram, which details the databases searched and articles excluded at each stage of the literature review, as well as the PRISMA checklist should be completed and included with all manuscript submissions. If observational studies are also included, the Meta-analyses of Observational Studies in Epidemiology (MOOSE) should also be used.5Stroup DF Berlin JA Morton SC et al.Meta-analysis of observational studies in epidemiology: A proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group.JAMA. 2000; 283: 2008-2012Crossref PubMed Scopus (16539) Google Scholar Although the focus of this article is not a detailed overview of systematic reviews, the following steps are essential to ensure a high-quality meta-analysis:(1)A trained researcher or medical librarian should participate in drafting the study protocol, selecting databases to be searched, and designing search terms.(2)Two or more investigators should independently review the search results and decide whether articles should be included or excluded. If disagreement occurs, a third investigator should resolve any disputes. Included and excluded studies should be tallied. Some studies are published as separate articles (e.g., 1 providing outcomes, another on toxicities) or as updates to data (e.g., 5-year update, 10-year update). For studies published as updates or as parts of another study, it is important to not double count patients. Double counting studies result in biased meta-analysis estimates.(3)For included studies, pertinent data should be extracted.(4)Authors should consider performing a risk of bias assessment for all included studies. Commonly used bias assessment tools are the Centre for Evidence-Based Medicine levels of evidence and the Newcastle-Ottawa Scale.6Howick J. Oxford Centre for Evidence-based Medicine - Levels of Evidence (March 2009). Available at: https://www.cebm.ox.ac.uk/resources/levels-of-evidence/oxford-centre-for-evidence-based-medicine-levels-of-evidence-march-2009. Accessed 24 December 2022.Google Scholar,7Lo CK Mertz D Loeb M Newcastle-Ottawa Scale: Comparing reviewers’ to authors’ assessments.BMC Med Res Methodol. 2014; 14: 45Crossref PubMed Scopus (1169) Google Scholar The Centre for Evidence-Based Medicine levels of evidence can be used for any type of study, ranging from opinion pieces and retrospective study designs to meta-analyses of RCTs. The Newcastle-Ottawa Scale can be used to assess the quality of observational studies. Additionally, funnel plots and sensitivity analyses should be employed to assess publication bias. In returning to the question asked in the vignette, Fig. 2A depicts the classic presentation of the results of a meta-analysis of phase 3 RCTs reporting 5-year rates of grade 2+ genitourinary toxicity in patients receiving hypofractionated IMRT for localized prostate cancer. The name of each study is present at the left in the “Study” column. The individual effect sizes are presented to the right. The 95% confidence intervals (CIs) presented are calculated from the proportion and patient counts from each individual study. In this example, the effect sizes are incidence proportions. The pooled effect sizes under the fixed and random-effects models and their associated 95% CIs are bolded and presented at the bottom of the figure, called a “forest plot,” which originates from the idea that these plots appear as a forest of lines.8Lewis S Clarke M. Forest plots: Trying to see the wood and the trees.BMJ. 2001; 322: 1479-1480Crossref PubMed Scopus (537) Google Scholar The forest plot depicts each of these effect sizes graphically.Finally, heterogeneity statistics specific to this forest plot are presented at the far left. We will provide a detailed overview of each of these components of meta-analyses in the forthcoming sections. Most meta-analyses in the medical literature use one of two models to calculate a pooled effect size: the fixed-effects model or the random-effects model. Mixed-effects models may also be employed that use regressions using study-level covariates. When discussing these different models, an important distinction should be made between the true effect size and the observed effect size. The true effect size is the effect size that occurs in an entire patient population and would be present if the study had an infinite sample size. As a result, no sampling error could occur, as every patient will be represented in the effect size calculation. In reality, the true effect size is not known because studies evaluate samples of participants from a population; therefore, some degree of sampling error is always present. The observed effect size is the effect size that is observed, which is based on a patient sample and does contain sampling error. For illustrative purposes, both fixed- and random-effects models have been used in Fig. 2A. The fixed-effects model assumes that all the included studies share a common true effect size and zero between-study variance. Thus, the true effect size should be identical across studies. However, because the sample sizes of each study are not infinite, the differences in effect sizes between studies are due to sampling error. The fixed-effects model is given by:Xi=μ+εiwhere Xi is the individual study effect size for study i, μ is the pooled effect size, and εi is the sampling error associated with study i and follows a normal distribution with mean 0 and variance σi2. The goal of fixed-effects meta-analysis is to weigh each study by the inverse of the variance for each individual study, because studies with smaller variances (and thus weighted heavier) presumably contribute an effect estimate closer to the true effect size. Thus, the weight for study i is calculated as follows:Wi=1σi2. The pooled effect size is then calculated by using a weighted mean, where each individual study effect size (Xi) is weighted by its corresponding study weight (Wi). As shown in Fig. 2A, the fixed-effects pooled effect size estimate is 17.9% (95% CI: 16.7%-19.1%). The assumption that each study has an identical true effect size is often not plausible in meta-analysis. Although the studies being combined in a meta-analysis share common attributes, it is uncommon for them to stem from one homogeneous population. Unlike fixed-effects models, random-effects models consider each study as a sample of a larger group of studies with some unknown, nonzero variance between them. As a result, each study has a different true effect size, which makes a fixed-effects model inappropriate. The true effect sizes are highly likely to differ between studies in the provided example in Fig. 2A. In each of these studies, patients with localized prostate adenocarcinoma received hypofractionated IMRT. Although it is reasonable to combine each study in a meta-analysis to obtain a pooled effect size, use of a fixed-effects model in this case would be an oversimplification. For example, on the MDACC trial, 28% and 71% of patients had low- and intermediate-risk prostate cancer, respectively.9Hoffman KE Voong KR Levy LB et al.Randomized trial of hypofractionated, dose-escalated, intensity-modulated radiation therapy (IMRT) versus conventionally fractionated IMRT for localized prostate cancer.J Clin Oncol. 2018; 36: 2943-2949Crossref PubMed Scopus (32) Google Scholar These patients underwent treatment to their prostate and proximal seminal vesicles. In the Fox Chase trial, 66% and 33% of patients had intermediate- and high-risk prostate cancer, respectively.10Pollack A Walker G Horwitz EM et al.Randomized trial of hypofractionated external-beam radiotherapy for prostate cancer.J Clin Oncol. 2013; 31: 3860-3868Crossref PubMed Scopus (383) Google Scholar Although the intermediate-risk patients were treated similarly to the MDACC patients, the high-risk patients underwent treatment to the entire seminal vesicles and pelvic lymph nodes. As a result, one would expect the Fox Chase trial patients to have a higher average rate of grade 2+ genitourinary toxicity. Therefore, it is likely that the true effect sizes differ between studies. The random-effects model assumes that the true effect size will differ between each of the included studies. Therefore, the variation in true effect sizes between each of the studies must be considered in addition to sampling errors. This model is given by:Xi=μ+εi+ωi,where ωi is the variation in effect sizes between μ and the true effect size for study i and follows a normal distribution with mean 0 and variance τ2. The other terms are defined the same as the fixed-effects model. The τ2 parameter refers to the between-study variance (often referred to as between study heterogeneity) and applies to all studies in a random-effects meta-analysis. There are multiple methods used to calculate τ2, most commonly the DerSimonian and Laird and restricted maximum likelihood (REML) methods. Methods for estimating τ2 have been extensively studied, as reported by Veroniki et al.11Veroniki AA Jackson D Viechtbauer W et al.Methods to estimate the between-study variance and its uncertainty in meta-analysis.Res Synth Methods. 2016; 7: 55-79Crossref PubMed Scopus (777) Google Scholar Although the Dersimonian and Laird method is the most commonly used, the REML method is the most frequently recommended.12Viechtbauer W. Confidence intervals for the amount of heterogeneity in meta-analysis.Stat Med. 2007; 26: 37-52Crossref PubMed Scopus (241) Google Scholar In Fig. 2A, the estimate of τ2 was calculated using the REML method as 0.03 (95% CI: 0.01-0.18). Study weights under the random-effects model are constructed in a similar fashion as those in the fixed-effects model, as the goal for both is to weigh each study by the inverse of its variance. However, τ2 must now be considered in this calculation:Wi=1σi2+τ2where Wi, σi2. and τ2 are the random-effects study weights, within-study variance, and between-study variance for study i, respectively. The pooled effect size is then calculated in an analogous fashion to the procedure used in the fixed effects model. As shown in Fig. 2A, the random-effects pooled effect size estimate is 26.2% (95% CI: 12.0%-43.7%). The choice of a fixed-effects or random-effects model is commonly made during the systematic review and study design phase of the meta-analysis.13Borenstein M Hedges LV Higgins JP Rothstein HR A basic introduction to fixed-effect and random-effects models for meta-analysis.Res Synth Methods. 2010; 1: 97-111Crossref PubMed Scopus (3482) Google Scholar It is also not an uncommon practice to select a random-effects model when there is significant study heterogeneity present; however, this is not the most robust method to account for study heterogeneity. When selecting a model, it is essential to consider the underlying study populations. In cases where each study was selected from an identical population, a fixed-effects model would likely be preferred. If the selected studies are composed of individuals from different populations, such as our provided example, then a random-effects model would likely be the best choice. The selection of the appropriate model depends on multiple factors, such as the research question of interest and individual study designs. As a result, it is common to perform post hoc analyses or sensitivity analyses to further assess the appropriateness of the chosen model.14Petitti DB. Approaches to heterogeneity in meta-analysis.Stat Med. 2001; 20: 3625-3633Crossref PubMed Scopus (359) Google Scholar Once the meta-analysis is completed, it is essential to assess the validity and generalizability of the pooled effect sizes. Two of the most common ways to do this are outlined in the forthcoming sections. The first is to assess the presence and extent of study heterogeneity. The second is to determine whether publication bias is present. Heterogeneity refers to the variation in true effect sizes in a meta-analysis. In the forthcoming subsections we review the Q-statistic, τ2, τ, and the I2 statistic, which are different ways of quantifying the study heterogeneity in a meta-analysis. It is important to note that there are additional heterogeneity quantification measures, such as the intraclass correlation coefficient and H-index; however, this article will not focus on these measures. The Q-statistic and its associated statistical test are common ways of assessing the presence or absence of heterogeneity in a meta-analysis. The Q-Statistic is the weighted sum of the squared deviations of each individual study effect size (Xi) from the pooled effect size (μ^), where the weight quantifies the contribution of each study and is given by:Q=∑i=1jWi(Xi−μ^)2 The Q-Statistic follows a χ2 distribution with j−1 degrees of freedom (df) under the null hypothesis (τ2=0), where j is the number of studies included in the meta-analysis. In Fig. 2A, the Q-statistic is presented as χ52 = 412.04. χ52 indicates that the Q-statistic in our example follows from a χ2 distribution with 5 df. To assess the presence of heterogeneity, a statistical test is performed, where a P value < 0.05 indicates that sufficient evidence exists to reject the null-hypothesis, which is the absence of study heterogeneity. In this example, the P value corresponding to the Q-statistic is < 0.01; thus we conclude that there is evidence of study heterogeneity. When interpreting Q-statistics and their associated P values, multiple important points must be considered. First, because these values are sensitive to the number of studies included, a nonsignificant P value does not mean that heterogeneity is absent, as this lack of significance may be due to low statistical power. Second, these values allow for a qualitative assessment of whether heterogeneity may be present and therefore cannot be used to quantify the magnitude of heterogeneity. The I2 statistic is a heterogeneity measure that is expressed as a percentage of unexplained variance ranging from 0% to 100%. Unlike τ, I2 does not depend on the scale of the effect sizes and therefore cannot be used to quantify deviations from the pooled effect size. Additionally, unlike the Q-statistic, it does not depend on the number of studies included. Rather, it is used to describe the percentage of heterogeneity present and is calculated using the following formula:I2=Q−(j−1)Qx100%where Q is the Q-Statistic and j is the number of studies included in the meta-analysis. In 2003, Higgins et al. provided common benchmarks that are used for I2 statistic interpretation. Cutoffs of 25%, 50%, and 75% indicate low, moderate, and substantial heterogeneity, respectively.15Higgins JP Thompson SG Deeks JJ Altman DG. Measuring inconsistency in meta-analyses.BMJ. 2003; 327: 557-560Crossref PubMed Scopus (43702) Google Scholar In Fig. 2A, the I2 statistic is 99%, which indicates the presence of substantial heterogeneity. Additionally, the Cochrane Handbook recommends four I2 interpretation ranges: 0% to 40% (may not be important), 30% to 60% (may represent moderate heterogeneity), 50% to 90% (may represent substantial heterogeneity), and 75% to 100% (may represent considerable heterogeneity). Unlike the Q-statistic and its associated P value, which depend on the number of studies in the meta-analysis, τ2 and τ do not depend on the number of studies included. In the previous section, we discussed τ2 and how it represents the between-study variances. In Fig. 2A, we see that the τ2has been calculated as 0.03 (95% CI: 0.01-0.18). However, because τ2 is a squared value and represents a variance, the standard deviation τ is often used because it is expressed in the same scale as the effect size metric. In Fig. 2A, by taking the square root of τ2 and its 95% CI, it follows that τ= 0.17 (95% CI: 0.10-0.24). This value serves as an estimate of the standard deviation of the pooled effect size, which is presented in Fig. 2A as 26.2% (95% CI: 12.0%-43.7%). Because τ is in the same scale as the individual study and pooled effect sizes, it serves as a useful way to inform the presence of heterogeneity. Therefore, because τ is in the same scale as the pooled effect size, when using τ to interpret the extent of heterogeneity it must be interpreted in the context of the magnitude of the pooled effect size.16Higgins JP Thompson SG. Quantifying heterogeneity in a meta-analysis.Stat Med. 2002; 21: 1539-1558Crossref PubMed Scopus (23806) Google Scholar Thus, in the provided example a τ= 17% compared with the random-effects pooled effect size of 26.2% would indicate the presence of a significant amount of heterogeneity. Although multiple methods exist to assess the presence of heterogeneity, there is no single method that should be used over the others. However, investigators commonly use the Q and I2 statistics together, and then use τ to directly quantify the variation between study effect sizes.16Higgins JP Thompson SG. Quantifying heterogeneity in a meta-analysis.Stat Med. 2002; 21: 1539-1558Crossref PubMed Scopus (23806) Google Scholar An overview of each of these heterogeneity statistics is provided in Table 1.Table 1Heterogeneity statistics used in meta-analysisStatisticNotesQ•Weighted sum of squared deviation of the observed effect size of each individual study and the pooled effect size•Statistical test based on the χ2 distribution with n – 1 degrees of freedom•Null hypothesis: the true treatment effect is identical across studies•Sensitive to the number of studies present•Provides a qualitative, not quantitative assessment of heterogeneityτ2•Between-study variances•Not sensitive to the number of studies present•Provides a quantitative assessment of heterogeneity•In the same scale as the variance of the effect sizes; therefore, difficult to interpret•Its square root, τ, is commonly usedτ•The square root of between-study variances, which is also the standard deviation of the between-study variance•Not sensitive to the number of studies present•Provides a quantitative assessment of heterogeneity•In the same scale as the effect sizes•Must be interpreted in context of the magnitude of the pooled effect sizeI2•The percentage of unexplained variance ranging from 0%-100%•Not sensitive to the number of studies present•Is not in the same scale as the effect sizes•Use of validated cutoffs can provide a descriptive assessment of the amount of heterogeneity present○Higgins et al:15Higgins JP Thompson SG Deeks JJ Altman DG. Measuring inconsistency in meta-analyses.BMJ. 2003; 327: 557-560Crossref PubMed Scopus (43702) Google Scholar 25% (low), 50% (moderate), 75% (substantial)○Cochrane Handbook:4Higgins JP, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Review of Interventions Version 6.2 (updated February 2021). Cochrane, 2021. Available at: https://training.cochrane.org/handbook/current. Accessed 7 January, 2022Google Scholar 0%-40% (may not be important), 30%-60% (may represent moderate heterogeneity), 50%-90% (may represent substantial heterogeneity), and 75%-100% (may represent considerable heterogeneity) Open table in a new tab In the medical literature, it is not uncommon for studies with positive results and/or larger effect sizes to be published more frequently than those without. As a result, a meta-analysis can become biased if these studies dominate the studies that are included, which is known as publication bias.17Zaorsky NG Wang X Lehrer EJ et al.Retrospective comparative effectiveness research: Will changing the analytical methods change the results?.Int J Cancer. 2022; 150: 1933-1940Crossref Scopus (4) Google Scholar Publication bias is a serious concern when conducting meta-analyses and can result in markedly biased pooled effect sizes. Thus, it is essential that investigators thoroughly assess the presence of publication bias. Figure 2B depicts the funnel plot from our provided example. In this example, the proportion of late grade 2+ genitourinary toxicity is plotted on the x-axis, with the standard errors plotted on the y-axis. The vertical line represents the pooled effect size, which in Fig. 2A is 26.2%. As the sample size of the study increases, the standard error decreases, as more study participants allow for greater statistical power when calculating estimates; therefore, larger studies appear toward the top of the plot and smaller studies appear toward the bottom. For example, the CHHiP trial enrolled 2151 patients who received HIMRT and appears at the top of the plot. The MDACC trial enrolled 102 patients and appears at the bottom. If publication bias is absent, the studies will be distributed symmetrically around the pooled effect size. If the studies were distributed asymmetrically, then publication bias may be present. The angled lines that form the left and right bounds of the triangle represent a 95% CI around the pooled effect size assuming a normal distribution. Visual inspection of a funnel plot alone is inadequate to conclude whether publication bias is present. Therefore, a statistical test is commonly used to further address this question. A commonly used method is the linear regression-based Egger's Test. In Fig. 2B, the P value corresponding to the Egger's Test is 0.12, which shows no sign of publication bias at the significance level of 0.05. This article primarily describes traditional meta-analysis. There are other types of meta-analysis with extensions to handle the data under different contexts, including individual patient data meta-analysis, network meta-analysis, multivariate meta-analysis if multiple endpoints of interest exist, and genomic data meta-analysis (e.g., microarray). Next, we will mainly focus on the first twowith more detailed illustration. Individual patient data (IPD) meta-analyses are a special type of meta-analysis, where instead of using study-level or summary data extracted from each individual study, the original research data for each patient are used. Individual patient data analyses can be very powerful and are considered the “gold standard” of systematic reviews. IPD meta-analysis can be conducted using 1-stage or 2-stage approaches.18Burke DL Ensor J Riley RD. Meta-analysis using individual participant data: One-stage and two-stage approaches, and why they may differ.Stat Med. 2017; 36: 855-875Crossref PubMed Scopus (293) Google Scholar The 1-stage approach analyzes IPD from all studies simultaneously in a single statistical analysis, and often uses hierarchical regression models with random effects to account for the clustering of participants within studies and any heterogeneity across studies. In the 2-stage approach, the aggregate data are estimated from each study separately, and then combined using traditional meta-analysis approaches. Although the 1-stage approach is the preferred method, the 2-stage approach is commonly used when IPD is missing from some, but not all, of the included trials. Readers are referred to Burke et al. (2017) for a comprehensive comparison of the two approaches.18Burke DL Ensor J Riley RD. Meta-analysis using individual participant data: One-stage and two-stage approaches, and why they may differ.Stat Med. 2017; 36: 855-875Crossref PubMed Scopus (293) Google Scholar A major advantage of IPD meta-analyses, especially the 1-stage approach, is that they are more powerful and flexible19Stewart GB Altman DG Askie LM Duley L Simmonds MC Stewart LA. Statistical analysis of individual participant data meta-analyses: A comparison of methods and recommendations for practice.PLoS One. 2012; 7: e46042Crossref PubMed Scopus (167) Google Scholar and avoid distributional assumptions of the effect size estimates for each study.20Riley RD Legha A Jackson D et al.One-stage individual participant data meta-analysis models for continuous and binary outcomes: Comparison of treatment coding options and estimation methods.Stat Med. 2020; 39: 2536-2555Crossref PubMed Scopus (17) Google Scholar Furthermore, they enable assessment of treatment effectiveness within subgroups of interest. However, these meta-analyses tend to be associated with significant expenditures and time resources and are therefore less common in the medical literature than study-level meta-analyses. Additionally, it is often difficult to obtain IPD for large numbers of studies. Traditional meta-analysis synthesizes direct, head-to-head comparisons. Network meta-analysis (NMA) compares multiple treatments simultaneously by combining direct and indirect comparisons into a network, for example, if direct evidence is available from RCTs comparing interventions A versus B and B versus C but RCTs comparing A and C are absent. NMA can combine the 2 direct comparisons and form a loop, or network, that compares all 3 treatments. One critical assumption in NMA is consistency, which means the agreement between the direct and indirect evidence. In the previous example, the effect size estimate of A versus C from indirect comparison is assumed to be the same as that from direct comparison. Thus, careful assessment of consistency in treatment comparisons is required. Multiple statistical methods, including both frequentist and Bayesian approaches, have been developed for NMA. Readers are referred to Efthimiou et al. (2016) for more details about methods and software implementations.21Efthimiou O Debray TP van Valkenhoef G et al.GetReal in network meta-analysis: A review of the methodology.Res Synth Methods. 2016; 7: 236-263Crossref PubMed Scopus (209) Google Scholar Meta-analysis is a powerful tool that can be used to increase statistical power, which can allow for detection of treatment effects with high precision that is often not possible with individual studies. It is essential that investigators adhere to strict methodology and carefully assess the quality of all studies that may be included. A detailed assessment of heterogeneity and publication bias must be performed to determine the validity and generalizability of any meta-analysis.