The future of Cochrane Neonatal
Roger F Soll,Colleen Ovelman,William McGuire
DOI: https://doi.org/10.1016/j.earlhumdev.2020.105191
Abstract:Cochrane Neonatal was first established in 1993, as one of the original review groups of the Cochrane Collaboration. In fact, the origins of Cochrane Neonatal precede the establishment of the collaboration. In the 1980's, the National Perinatal Epidemiology Unit at Oxford, led by Dr. Iain Chalmers, established the "Oxford Database of Perinatal Trials" (ODPT), a register of virtually all randomized controlled trials in perinatal medicine to provide a resource for reviews of the safety and efficacy of interventions used in perinatal care and to foster cooperative and coordinated research efforts in the perinatal field [1]. An effort that was clearly ahead of its time, ODPT comprised four main elements: a register of published reports of trials; a register of unpublished trials; a register of ongoing and planned trials; and data derived from pooled overviews (meta-analyses) of trials. This core effort grew into the creation of the seminal books, "Effective Care in Pregnancy and Childbirth" as well as "Effective Care of the Newborn Infant" [2,3]. As these efforts in perinatal medicine grew, Iain Chalmers thought well beyond perinatal medicine into the creation of a worldwide collaboration that became Cochrane [4]. The mission of the Cochrane Collaboration is to promote evidence-informed health decision-making by producing high-quality, relevant, accessible systematic reviews and other synthesized research evidence (www.cochrane.org). Cochrane Neonatal has continued to be one of the most productive review groups, publishing between 25 tpo to 40 new or updated systematic reviews each year. The impact factor has been steadily increasing over four years and now rivals most of the elite journals in pediatric medicine. Cochrane Neonatal has been a worldwide effort. Currently, there are 404 reviews involving 1206 authors from 52 countries. What has Cochrane done for babies? Reviews from Cochrane Neonatal have informed guidelines and recommendations worldwide. From January 2018 through June 2020, 77 international guidelines cited 221 Cochrane Neonatal reviews. These recommendations have included recommendations of the use of postnatal steroids, inhaled nitric oxide, feeding guidelines for preterm infants and other core aspects of neonatal practice. In addition, Cochrane Reviews has been the impetus for important research, including the large-scale trial of prophylactic indomethacin therapy, a variety of trials of postnatal steroids, trials of emollient ointment and probiotic trials [6]. While justifiably proud of these accomplishments, one needs to examine the future contribution of Cochrane Neonatal to the neonatal community. The future of Cochrane Neonatal is inexorably linked to the future of neonatal research. Obviously, there is no synthesis of trials data if, as a community, we fail to provide the core substrate for that research. As we look at the current trials' environment, fewer randomized controlled trial related to neonates are being published in recent years. A simple search of PubMed, limiting the search to "neonates" and "randomized controlled trials" shows that in the year 2000, 321 randomized controlled trials were published. These peaked five years ago, in 2015, with close to 900 trials being published. However, in 2018, only 791 studies are identified. Does this decrease represent a meaningful change in the neonatal research environment? Quite possibly. There are shifting missions of clinical neonatology at academic medical institutions, at least in the United States, with a focus on business aspects as well as other important competing clinical activities. Quality improvement has taken over as one of the major activities at both private and academic neonatal practices. Clearly, this is a needed improvement. All units at levels need to be dedicated to improving the outcomes of the sick and fragile population we care for. However, this need not be at the expense of formal clinical trials. It is understandable that this approach would be taken. Newer interventions frequently relate to complex systems of care and not the simple single interventions. Even trials that might traditionally have been done as randomized controlled trials, such as the introduction of a new mode of ventilation, are in reality complex challenges to the ability of institutions to create systems to adapt to these new technologies. Cost of doing trials has always been a barrier. The challenging regulatory and ethical environment contributes to these problems as well [7]. Despite these barriers, how does the research agenda of the neonatal community move forward in the 21st Century? We need to reassess how we create and disseminate our research findings. Innovative trial designs will allow us to address complex issues that we may not have tackled with conventional trials. Adaptive designs may allow us to look at potentially life-saving therapies in a way that feel more efficient and more ethical [8]. Clarifying issues such as the use of inhaled nitric oxide in preterm infants would be greatly served if we even knew whether or not there are hypoxemic preterm infant who would benefit from this therapy [9]. Current trials do not suggest so, yet current practice tells us that a significant number of these babies will receive inhaled nitric oxide [10-13]. Adaptive design, such as those done with trials of extracorporeal membrane oxygenation (ECMO), would allow us to quickly assess whether, in fact, these therapies are life-saving and allow us to consider whether or not further trials are needed [14,15]. Our understanding that many interventions involve entire systems approaches does not relegate us only to doing quality improvement work. Cluster designs may allow us to test more complex interventions that have usually been under the purview of quality improvement [16-18]. Cluster trials are well suited for such investigations and can be done with the least interruption to ongoing care. Ultimately, quality improvement is the application of the best evidence available (evidence-based medicine is "what to do" and evidence-based practice is "how to do"). [19,20]. Nascent efforts, such as the statement on "embedding necessary research into culture and health" (the ENRICH statement) call for the conduct of large, efficient pragmatic trials to evaluate neonatal outcomes, as in part called for in the ALPHA Collaboration [21,22]. This statement envisions an international system to identify important research questions by consulting regularly with all stakeholders, including patients, public health professionals, researchers, providers, policy makers, regulators, funders of industry. The ENRICH statement envisions a pathway to enable individuals, educational institutions, hospitals and health-care facilities to confirm their status as research-friendly by integrating an understanding of trials, other research and critical thinking and to teaching learning and culture, as well as an engagement with funders, professional organizations and regulatory bodies and other stake holders to raise awareness of the value of efficient international research to reduce barriers to large international pragmatic trials and other collaborative studies. In the future, if trials are to be done on this scale or trials are prospectively designed to be analyzed together, core outcome measures must be identified and standardized. That clinical trials supply estimates of outcomes that are relevant to patients and their families is critical. In addition, current neonatal research evaluates many different outcomes using multiple measures. A given measure can have multiple widely used definitions. Bronchopulmonary dysplasia (or chronic lung disease just to add to the confusion) quickly comes to mind [23,24]. The use of multiple definitions when attempting to measure the same outcome prevents synthesis of trial results and meta-analysis and hinders efforts to refine our estimates of effects. Towards that end, Webbe and colleagues have set out to develop a core outcome set for neonatal research [25]. Key stakeholders in the neonatal community reviewed multiple outcomes reported in neonatal trials and qualitative studies. Based on consensus, key outcome measures were identified, including survival, sepsis, necrotizing enterocolitis, brain injury on imaging, retinopathy or prematurity, gross motor ability, general cognitive ability, quality of life, adverse events, visual impairment or blindness, hearing impairment or deafness, chronic lung disease/bronchopulmonary dysplasia. Trials registration has to be a continued focus of the neonatal community. Trials registration allows for systematic reviewers to understand whether or not reporting bias has occurred [26]. It also allows for transparent incorporation of these core outcome measures. Ultimately, trials registration should include public reporting of all of these core outcomes and, in the future, access to data on an individual level such that more sophisticated individual patient data meta-analysis could occur. Lastly, there is no reason to see clinical trials and quality improvement as separate or exclusive activities. In fact, in the first NICQ Collaborative, conducted by Vermont Oxford Network, participation in a trial of postnatal steroids was considered part of the quality improvement best practices as opposed to simply choosing an as-of-yet unproven approach to use of this potent drug [27]. What role will Cochrane Neonatal play as we move forward in the 21st Century? As the neonatal community moves forward with its' research agenda, Cochrane Neonatal must not only follow but also lead with innovative approaches to synthesizing research findings. Cochrane Neonatal must continue to work closely with guideline developers. The relationship between systematic review production and guideline development is clearly outlined in reports from the Institute of Medicine [28,29]. Both are essential to guideline development; the systematic review group culling the evidence for the benefits and harms of a given intervention and the guideline group addressing the contextual issues of cost, feasibility, implementation and the values and preferences of individuals and societies. Most national and international guidelines groups now routinely use systematic reviews as the evidence basis for their guidelines and recommendations. Examples of the partnership between Cochrane Neonatal and international guideline development can be seen in our support of the World Health Organization (WHO) guidelines on the use of vitamin A or the soon to be published recommendations from the International Liaison Committee on Resuscitation (ILCOR) on cord management in preterm and term infants [30]. In the future, we need to collaborate early in the guideline development process so that the reviews are fit for purpose and meet the needs of the guideline developers and the end users. Towards this end, all Cochrane Neonatal reviews now contain GRADE assessments of the key clinical findings reported in the systematic review [31]. Addition of these assessments addresses the critical issue of our confidence in the findings. We are most confident in evidence provided by randomized controlled trials but this assessment can be can be downgraded if the studies that reported on the outcome in question had a high risk of bias, indirectness, inconsistency of results, or imprecision, or where there is evidence of reporting bias. Information provided by GRADE assessments is seen as critical in the process of moving from the evidence to formal recommendations [32]. We need to explore complex reviews, such as network (NMA) or multiple treatment comparison (MCT) meta-analyses, to address issues not formally addressed in clinical trials [33]. In conditions where there are multiple effective interventions, it is rare for all possible interventions to have been tested against each other [34]. A solution could be provided by network meta-analysis, which allows for comparing all treatments with each other, even if randomized controlled trials are not available for some treatment comparisons [34]. Network meta-analysis uses both direct (head-to-head) randomized clinical trial (RCT) evidence as well as indirect evidence from RCTs to compare the relative effectiveness of all included interventions [35]. However, Mills and colleagues note that the methodological quality of MTCs may be difficult for clinicians to interpret because the number of interventions evaluated may be large and the methodological approaches may be complex [35]. Cochrane Neonatal must take a role in both the creation of such analyses and the education of the neonatal community regarding the pitfalls of such an approach. The availability of individual patient data will make more sophisticated analyses more available to the community. Although the current crop of individual patient data meta-analyses (including the reviews of elective high frequency ventilation, inhaled nitric oxide and oxygen targets) have not differed substantially from the findings of the trials level reviews (suggesting that, in fact, sick neonates are more alike that unalike), there still will be a large role for individual patient data meta-analysis, at least to end the unfound conclusions that these therapies are effective in various subgroups (be it issues of sex, disease severity, or clinical setting) [36-39]. Future trials should take a lesson from the NeOProM Collaborative [37,39]. Given the difficulty in generating significant sample size and creating funding in any single environment, trials with similar protocols should be conducted in a variety of healthcare settings with an eye towards both study level and individual patient level meta-analysis at the conclusion of those trials, allowing for broader contribution to the trials data, more rapid accrual of sample size, and more precise results. We need to educate the neonatal community regarding the use and abuse of diagnostic tests. Diagnostic tests are a critical component of healthcare but also contribute greatly to the cost of medical care worldwide. These costs include the cost of the tests themselves and the costs of misdiagnosis and treatment of individuals who will not benefit from those treatments. Clinicians may have a limited understanding of diagnostic test accuracy, the ability of a diagnostic test to distinguish between patients with and without the disease or target condition [41,42]. Efforts such as Choosing Wisely have tried to identify these deficiencies [40]. As Cochrane has increased the general literacy of both the medical and general population regarding the interpretation of the results of interventions on various diseases, so should Cochrane move forward and improve the understanding of diagnostic testing. We need to become more efficient at creating and maintaining our reviews. The time spent to produce systematic reviews is far too great. In average, it takes between 2½ to 6½ years to produce a systematic review, requiring intense time input for highly trained and expensive experts. Innovations in the ways in which we produce systematic reviews can make the review process more efficient by outsourcing some of the tasks or crowdsourcing to machine learning. We need to let the crowd and machine learning innovations help us sort the massive amounts of information needed to conduct systematic reviews. It can also allow for "live" updating of critical reviews where the research landscape is quickly changing [43]. Lastly, Cochrane Neonatal must focus more on users of the reviews and not necessarily authors of the reviews. Current Cochrane programming speaks of Cochrane training with an eye towards developing the skills of individuals who will conduct systematic reviews. While this is clearly needed and laudable, the fact of the matter is that most of the community will be "users" of the reviews. Individuals who need to understand how to use and interpret the findings of systematic reviews. These review users include clinicians, guideline developers, policy makers and families. Incorporation of GRADE guidelines has been a huge step in adding transparency to the level of uncertainty we have in our findings. From a family's perspective, we need to overcome the environment of mistrust or misunderstanding of scientific evidence and how we convey what we know, and our uncertainty about what we know, to parents and families.