Reporting Assessment of Multicenter Clinical Trial Protocols: A Cross-Sectional Study

Xuan Zhang,Nana Wang,Lin Zhang,Chung Wah Cheng,Ziyi Liu,Feng Liang,Weifeng Xiong,Jiashuai Deng,Dongni Shi,Wanting Cui,Yanfang Ma,Taixiang Wu,Chun Pong Chan,Aiping Lyu,Chen Yao,Zhaoxiang Bian
DOI: https://doi.org/10.1111/jebm.12519
2023-01-01
Abstract:Journal of Evidence-Based MedicineEarly View LETTEROpen Access Reporting assessment of multicenter clinical trial protocols: A cross-sectional study Xuan Zhang, Xuan Zhang orcid.org/0000-0002-9230-9210 Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, China Centre for Chinese Herbal Medicine Drug Development Limited, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorNana Wang, Nana Wang Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorLin Zhang, Lin Zhang College of Chinese Medicine, Tianjin University of Traditional Chinese Medicine, Tianjin, ChinaSearch for more papers by this authorChung Wah Cheng, Chung Wah Cheng Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorZiyi Liu, Ziyi Liu Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorFeng Liang, Feng Liang Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorWeifeng Xiong, Weifeng Xiong College of Chinese Medicine, Beijing University of Chinese Medicine, Beijing, ChinaSearch for more papers by this authorJiashuai Deng, Jiashuai Deng College of Chinese Medicine, Beijing University of Chinese Medicine, Beijing, ChinaSearch for more papers by this authorDongni Shi, Dongni Shi Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorWanting Cui, Wanting Cui College of Chinese Medicine, Tianjin University of Traditional Chinese Medicine, Tianjin, ChinaSearch for more papers by this authorYanfang Ma, Yanfang Ma Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, China Centre for Chinese Herbal Medicine Drug Development Limited, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorTaixiang Wu, Taixiang Wu Chinese Cochrane Centre, West China Hospital, Sichuan University, China Trial Registration Center, Chengdu, ChinaSearch for more papers by this authorChun Pong Chan, Chun Pong Chan Faculty of Science, Department of Computer Science, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorAiping Lyu, Aiping Lyu Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorChen Yao, Chen Yao Peking University Clinical Research Institute, Peking University First Hospital, Beijing, ChinaSearch for more papers by this authorZhaoxiang Bian, Corresponding Author Zhaoxiang Bian [email protected] Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, China Centre for Chinese Herbal Medicine Drug Development Limited, Hong Kong Baptist University, Hong Kong, China Correspondence Zhaoxiang Bian, Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, China; Centre for Chinese Herbal Medicine Drug Development Limited, Hong Kong Baptist University, Hong Kong, China. Email: [email protected]Search for more papers by this author Xuan Zhang, Xuan Zhang orcid.org/0000-0002-9230-9210 Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, China Centre for Chinese Herbal Medicine Drug Development Limited, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorNana Wang, Nana Wang Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorLin Zhang, Lin Zhang College of Chinese Medicine, Tianjin University of Traditional Chinese Medicine, Tianjin, ChinaSearch for more papers by this authorChung Wah Cheng, Chung Wah Cheng Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorZiyi Liu, Ziyi Liu Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorFeng Liang, Feng Liang Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorWeifeng Xiong, Weifeng Xiong College of Chinese Medicine, Beijing University of Chinese Medicine, Beijing, ChinaSearch for more papers by this authorJiashuai Deng, Jiashuai Deng College of Chinese Medicine, Beijing University of Chinese Medicine, Beijing, ChinaSearch for more papers by this authorDongni Shi, Dongni Shi Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorWanting Cui, Wanting Cui College of Chinese Medicine, Tianjin University of Traditional Chinese Medicine, Tianjin, ChinaSearch for more papers by this authorYanfang Ma, Yanfang Ma Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, China Centre for Chinese Herbal Medicine Drug Development Limited, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorTaixiang Wu, Taixiang Wu Chinese Cochrane Centre, West China Hospital, Sichuan University, China Trial Registration Center, Chengdu, ChinaSearch for more papers by this authorChun Pong Chan, Chun Pong Chan Faculty of Science, Department of Computer Science, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorAiping Lyu, Aiping Lyu Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, ChinaSearch for more papers by this authorChen Yao, Chen Yao Peking University Clinical Research Institute, Peking University First Hospital, Beijing, ChinaSearch for more papers by this authorZhaoxiang Bian, Corresponding Author Zhaoxiang Bian [email protected] Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, China Centre for Chinese Herbal Medicine Drug Development Limited, Hong Kong Baptist University, Hong Kong, China Correspondence Zhaoxiang Bian, Chinese Clinical Trial Registry (Hong Kong), Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, China; Centre for Chinese Herbal Medicine Drug Development Limited, Hong Kong Baptist University, Hong Kong, China. Email: [email protected]Search for more papers by this author First published: 03 March 2023 https://doi.org/10.1111/jebm.12519AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Randomized controlled trials (RCTs) are the top evidence for effectiveness research of a new intervention or treatment. Multicenter design is commonly used in RCTs to increase the sample size and improve the external validity, particularly in Phase II or III studies.1 With the globalization of drug development, increasing emphasis is being placed on multicenter randomized controlled trials (MRCTs).2 However, MRCTs reach their full potential only if they are designed, conducted, and reported appropriately. In our previous survey in 2021, we have analyzed 2844 final reports of MRCTs during 1975–2019 and found that several multicenter-related information was absent or incompletely provided in their publications. To investigate the reason for unsatisfactory reporting level of MRCTs, we also identified that only 19% studies provided their trial protocol that can be accessed, and 11% descripted the consistency or disparity of protocols across centers.3 Given the importance of trial protocols, an international group of stakeholders developed the SPIRIT 2013 Statement (Standard Protocol Items: Recommendations for Interventional Trials) to provide a basic guidance for the reporting of protocol.4, 5 However, it is unclear whether the reporting of published MRCT protocols are completeness according to the SPIRIT. Furthermore, no previous study has assessed whether multicenter-related aspects are fully considered in these protocols. In particular, some issues include the criteria and responsibilities of participating centers, the quality control regarding administration of intervention(s) and measurements of outcome(s) across centers, the plans of data collection, management and monitoring across centers, and the heterogeneity analysis in the statistical analysis plan (SAP) that are critical for promoting transparency of an MRCT. Therefore, this review aimed to evaluate the reporting characteristics of MRCT protocols based on the SPIRIT 2013 checklist and on a specially designed multicenter-specific checklist. In this review, we defined the protocol of an MRCT as a randomized controlled clinical trial where the data of participants will be collected from more than one center. Accordingly, MRCT protocols published in English or Chinese up to 11 November 2022 were included with no restriction to conditions, interventions, and controls. Except for the exclusion of interim analysis, results publications, single-center studies, nonrandomized or noncontrolled studies, reviews, case reports, and nonhuman studies, we also exclude repeat records, protocols not published as full text (such as abstracts, full reports cannot be accessed or withdrawn), and those unavailable in English or Chinese language. A systematic search was conducted in the following databases: (1) English databases: Ovid of All EBM Reviews, Embase, and Ovid MEDLINE(R). Some duplicates or nonfull text were preliminary removed by Ovid. (2) Chinese databases: CNKI, VIP, and Wanfang. The original search time was up to 7 July 2021, and an updated search to 11 November 2022. The restriction of language was English or Chinese. Search strategy is presented in Supplementary Table S1. Two reviewers (ZYL and LZ) independently screened the titles and abstracts of the retrieved records based on the inclusion and exclusion criteria. Full reports of any potentially relevant papers were reviewed for further assessment of eligibility. Disagreements were settled via discussion or by consulting a third reviewer (XZ). Data extraction was conducted by two reviewers (ZYL and NNW) independently using a predesigned form, which contained the following elements: types of study design (e.g., assignment, randomization, blinding, and sample size), features of participating centers (e.g., number and distribution of centers, international or national trials), types of diseases based on ICD-11, categories of intervention(s) and control(s), funding sources, and trial registration. The reporting quality was evaluated according to (1) the SPIRIT 20136 and (2) a special-designed multicenter checklist, which was developed by four researchers (ZXB, CY, XZ, and TXW) based on discussion (Table 1). Scoring rules are provided in Supplementary Tables S2 and S3. Generally, each item/question was scored in terms of two possibilities: “1” for “fully reported” or “not applicable,” “0” for “insufficiently reported” or “unreported,” Six reviewers (ZYL, FL, JSD, WTC, CWC, and NNW) were trained and participated the evaluations. During the process, the quality assessment of each protocol was conducted by one author and verified by another author. Possible disagreements were resolved with the consultation of a third senior review author (XZ or ZXB). TABLE 1. Questions for assessing the reporting of multicenter-specific items. Questions Specifics Q1 Whether the multicenter design was identified in the title? Q2 Whether the multicenter design was reported in the Abstract? Q3 Whether relevant rationale for using a multicnter design was provided in the protocol? Q4 Whether the multicenter was specified in the trial design section? Q5 Whether any description of participating center was provided? Such as the namelist of study centers or the selection criteria of centers, etc? Q6 Whether any description of methods used to ensure that interventions will be administered consistent across centers was provided? Or indicated any variations across centers? Q7 Whether any description of intervention training arrangement across centers? Q8 Whether any description of methods to ensure consistent outcome measurements across centers was provided? Q9 Whether any preallocation of sample size in each center was mentioned? Q10 Whether any description of recruitment strategies across centers was provided? Such as the recruitment ways, schedule, or timeline, etc. Q11 Whether the allocation sequence was specified as centralized or stratified by center? Q12 Whether any description of methods for data collection across centers was provided? Such as central system used to ensure consistent among centers, or indicated any differences across centers? Q13 Whether any statistical analysis plan for the heterogeneity assessment and adjustment for central effect was provided? Q14 Whether any description of auditing, such as conducted for each center or centrally was provided? We applied frequency and percentage to present categorical variables and median and interquartile range (IQR) to present continuous variables. For individual item of reporting quality, the compliance rate was calculated with the number of items acquired “1” based on the total number of included reports, which was further categorized as three levels: excellent compliance (>90%), good compliance (between 65% and 90%), and poor compliance (<65%).7-9 For the overall items of checklists, the reporting score was recorded as mean and standard deviations (SD). As the SPIRIT was published in 2013, we divided the included protocols as publication year before and after 2013. Also, as the number of publications are increased rapidly in recent 3 years, we further set up two groups in the column of “after 2013” to achieve the sample balance. Thus, three subgroups for comparisons are “2005-2012,” “2013-2019,” and “2020-Nov 2022,” All data were collected and recorded in Microsoft Office Excel (Version 2016). Statistics analysis were performed using SPSS software, version 25.0. A flowchart of article selection is shown in Supplementary Figure S1. Briefly, a total of 1191 protocols were finally included. Of these, five articles (0.4%) published in Chinese and 1186 (99.6%) in English. The included 1191 MRCT protocols were published between January 2005 to 11 November 2022. The most common design was a parallel-group assignment (86.2%) of pharmaceutical treatments (35.6%) and active control (62.2%). 58.7% trials adopted blinding and 30.8% used central randomization (including stratified by centers). The three conditions most commonly studied were Neoplasms (9.7%); diseases of the circulatory system (9.6%); and mental, behavioral, or neurodevelopmental disorders (9.3%). The included protocols presented a median of seven participating centers, a median of 242 sample size, and a median of three participating countries for 187 (15.7%) international MRCTs. The included MRCTs were distributed in 88 countries, of which the most common countries are China (17.2%), the United Kingdom (8.9%), and the United States (7.1%). Most trials (>94%) reported whether has funding and trial registration or not, but 95.5% were published in the journals with IF < 6. Of 1191 included trials, 53.7% submitted research ethics approval from one central committee while 26.4% from local committees of each center. 39.0% protocols reported the collection plan of biological specimens. Details are shown in Supplementary Tables S4–S6. For the quality assessment, detailed results are presented in Supplementary Tables 7 and 8. In summary, the mean (SD) reporting score of the SPIRIT was 42.8 (5.9). Specifically, the quality of reporting was excellent (>90%) in 25 items (1, 2a, 2b, 3, 4, 5a, 5b, 6a, 7, 9, 10, 11a, 12, 13, 14, 16a, 16b, 20a, 24, 26a, 26b, 28, 31b, and 32); good (65-90%) in 18 items (5c, 5d, 6b, 8, 11b, 15, 16c, 17a, 18b, 19, 20b, 21a, 22, 23, 25, 27, 29, and 31a); and poor (< 65%) in 8 items (11c, 11d, 17b, 20c, 21b, 30, 31c, and 33). However, the mean reporting score of the multicenter-specific items was 8.3 (2.3), markedly low. No item was “excellent,” and the “good” reporting in eight items (Q1, Q2, Q4, Q5, Q6, Q11, Q12, and Q14). The remaining six items were reported poorly (Q3, Q7, Q8, Q9, Q10, and Q13), of which three items showed extremely low (<30%), covering the rationale for using a multicenter design, methods used to ensure consistent intervention administrated and outcomes measurements across centers, sample size preallocation in each center, and statistical analysis plan for heterogeneity assessment. Protocol publications in later years were associated with increased reporting quality, especially after 2013. The reporting scores increased significantly from 39.5 (7.1) during 2005–2012 to 42.5 (6.1) during 2013–2019, further increased slightly to 42.8 (5.5) in recent three years, out of 51 in total. However, it is noted that the scores for multicenter specifics were still quite low 8.3 (2.3), out of 14 in total, even a mild decrease during 2020-Nov 2022 was identified as 8.2 (2.3). This study identified that the current reporting of MRCT protocol publications is not optimal. Given the increasing feasibility of performing MRCTs, the extraordinary value of their findings to patients and science, and the expense of these trials, improving reporting in the phase of study protocol should be a priority. Developing standard reporting items specifically relevant to multicenter trials as an extension to the general SPIRIT statement might be an expedient, efficient, and effective means to achieving the improvement needed.10 AUTHOR CONTRIBUTIONS ZXB conceived and designed the study; XZ drafted the manuscript; LZ, ZYL, and FL searched and screened the literatures; FL, ZYL, WFX, NNW, JSD, WTC, DNS, YFM, and CWC extracted, assessed, and checked the data; LZ, NNW, CPC, and XZ performed and checked the statistical analysis; TXW, APL, and CY provided critical comments for this research; XZ and ZXB revised and finalized the manuscript. All authors approved the final version of the manuscript. FUNDING This work was supported by the Health and Medical Research Fund (No. 18192671), Hong Kong, China; Chinese Medicine Development Fund (No. 20B2/027A), Hong Kong, China; and China Center for Evidence Based Traditional Chinese Medicine, CCEBTM (2020YJSZX-5). The funders had no role in the design of the study, in the collection, analysis, and interpretation of data, nor in the writing of the manuscript. Supporting Information Filename Description jebm12519-sup-0001-SuppMat.docx912 KB Supplementary Information Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article. REFERENCES 1Chung KC, Song JW. A guide to organizing a multicenter clinical trial. Plast Reconstr Surg. 2010; 126(2), 515– 523. 2Huang Q, Chen G, Yuan Z, et al. Design and sample size considerations for simultaneous global drug development program. J Biopharm Stat. 2012; 22(5), 1060– 1073. 3Zhang X, Lam WC, Liu F, et al. A Cross-sectional literature survey showed the reporting quality of multicenter randomized controlled trials should be improved. J Clin Epidemiol. 2021; 137: 250– 261. 4Dane A, Klein Gebbink A-S, Van Der Kuy PHM. The importance of publishing research protocols for pharmacoeconomic studies. Eur J Hosp Pharm. 2023; 30(1): e4. 5Chan An-W, Tetzlaff JM, Altman DG, et al. SPIRIT 2013 Statement: Defining standard protocol items for clinical trials. Ann Intern Med. 2013; 158(3), 200– 207. 6Chan An-W, Tetzlaff JM, Gotzsche PC, et al. SPIRIT 2013 Explanation and Elaboration: Guidance for protocols of clinical trials. BMJ. 2013; 346:e7586. 7Stevanovic A, Schmitz S, Rossaint R, et al. CONSORT item reporting quality in the top ten ranked journals of critical care medicine in 2011: a retrospective analysis. PLoS One. 2015; 10(5): e0128061. 8Xia Y, Chen YL, Zeng Z, et al. Using the RIGHT statement to evaluate the reporting quality of clinical practice guidelines in traditional Chinese medicine. PLoS One. 2018; 13(11): e0207580. 9Zhang X, Zhang L, Xiong W, et al. Assessment of the reporting quality of randomised controlled trials of massage. Chin Med. 2021; 16(1): 64. 10 Enhancing the QUAlity and Transparency Of health Research (EQUAOR) Network. Available at: https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-clinical-trials-protocols/ #MCT Early ViewOnline Version of Record before inclusion in an issue ReferencesRelatedInformation
What problem does this paper attempt to address?