Network Meta-Analyses—Better Than Nothing?

Massimo Di Maio

DOI: https://doi.org/10.1001/jamanetworkopen.2024.1292

2024-03-08

JAMA Network Open

Abstract:In the last few years, immune checkpoint inhibitors have shown positive results in the treatment of patients with resectable non–small cell lung cancer (NSCLC), while until recently the only systemic treatment that had demonstrated a significant benefit compared with surgery alone was chemotherapy, administered as adjuvant treatment (more often) or as neoadjuvant therapy (less commonly). 1 In detail, some recent trials evaluated the use of adjuvant immunotherapy: 1 trial evaluated the simple addition of immunotherapy to preoperative treatment without further postoperative systemic therapies, while other studies tested as experimental treatment the addition of immunotherapy both before and after surgery. Unfortunately, the latter design does not allow us to understand the benefit, if any, associated with the postoperative administration of immunotherapy to patients who have already received the same drug as part of the neoadjuvant treatment. In their systematic review and network meta-analysis (NMA), Zhou and colleagues 2 explored whether the administration of immune checkpoint inhibitors before and after surgery would result in better outcomes than the administration of the same therapy entirely before surgery to patients with resectable NSCLC. According to their results, the addition of programmed cell death 1 (PD-1) and programmed death ligand 1 (PD-L1) inhibitors in the adjuvant phase to neoadjuvant treatment with PD-1 or PD-L1 inhibitors and chemotherapy was not associated with a clinically meaningful and significant improvement in event-free survival or overall survival—the latter end point being limited by a low maturity level—for patients with resectable NSCLC, while the longer therapy was associated with an increased incidence of treatment-related adverse events. However, as in most NMAs, the authors concluded that "future validation of these findings is warranted through head-to-head randomized clinical trials." 2 In this concluding sentence lies all the charm and at the same time the intrinsic limitation of NMAs; do they really add useful, reliable, and definitive information or not? When more than 1 treatment strategy has been used and compared in the same clinical setting, NMAs (which are also known as multiple treatment comparison meta-analyses , indirect meta-analyses , or mixed treatment meta-analyses ) offer a technical method to compare the relative effectiveness of all included interventions, allowing us to synthesize and interpret all the available evidence. 3 ,4 However, differently from "traditional" meta-analyses, NMAs, by definition, are not restricted to combining the results of studies that directly compared treatments of interest; rather, they compare the results of 2 or more treatments, combining studies that have a common treatment arm (in this case, the control arm with neoadjuvant chemotherapy before surgery). Some years ago, the Italian Association of Medical Oncology produced a methodological document about the choice among different treatments approved for the same therapeutic indication. 5 In that document, the working group stated that indirect comparisons are, in many cases, unreliable and that from a methodological point of view, such meta-analyses represent a potentially dangerous tool because they dress up with methodological rigor and objectivity comparisons that remain weak and scientifically debatable. Nonetheless, the panel emphasized that indirect comparisons can be useful in hypothesis generation and should be "an exercise at the highest scientific level, in which statistical and clinical skills are integrated in an attempt to examine the distribution of the effects of the various treatments." In general, NMAs should serve as support and not as a decisional tool to sanction the superiority of one treatment strategy over another, and the interpretation of the indirect comparisons should be limited to the discussion of differences of medium or large magnitude among treatments, ignoring the differences of modest clinical relevance, in which the weight of biases may be actually larger than the true difference between the treatments. 5 If we apply this "prudential" approach to the setting explored in the meta-analysis by Zhou and colleagues, 2 we can conclude that, to be adopted in clinical practice, postoperative immunotherapy—which is associated with higher rates of toxic effects and higher costs—should demonstrate a clear medium to large benefit in patient outcomes. At the moment, this is not the case because—based on the available evidence synthesized by the NMA—we can reasonably exclude that the addition of postoperative immunotherapy is associated with such a large improvement. In my opinion, beyond the intrinsic limitations of NMAs, the comparison of strategies addressed by the work of Zhou and colleagues 2</sup -Abstract Truncated-

medicine, general & internal