“Third Eye” of Colonoscopists: a Preclinical Comparative Study of Artificial Intelligence-Assisted Polyp Detection
Xinjue He,Lei Xu,Chengfu Xu,Jie Zhang,Meng Li,Weihong Wang,Xiaoqing Ding,Youming Li,Xuequn Zhang
DOI: https://doi.org/10.1097/cm9.0000000000002364
2022-01-01
Abstract:To the Editor: In China, colorectal cancer (CRC) ranks third in cancer incidence and fifth in cancer-related deaths.[1] Most CRCs arise from adenomatous polyps. Colonoscopy can detect and remove polyps early to prevent CRC. However, the polyp miss rate in conventional colonoscopy may be more than 40%.[2] New endoscopic devices for detecting polyps have made some progress over the past few decades,[3] but are still operator-dependent and require additional costs. Therefore, artificial intelligence (AI)-assisted polyp detection systems have emerged in recent years. This study aimed to evaluate the effectiveness of our newly developed AI-assisted polyp detection system for colonoscopy in a preclinical manner. We conducted a controlled study comparing the polyp detection results of AI video analysis and conventional colonoscopy. Adult patients scheduled for colonoscopy at Ningbo Hospital of Zhejiang University were considered for inclusion. The exclusion criteria were emergency colonoscopy, colonoscopic polypectomy, history of colorectal polyps, and patients with severe intestinal diseases or contraindications for colonoscopy. This study is a part of a multi-center project including Ningbo Hospital and the First Affiliated Hospital, College of Medicine, Zhejiang University; this protocol was approved by the Research Ethics Committee of the First Affiliated Hospital, College of Medicine, Zhejiang University (IRB No. 2018-524). The informed consent was waived. Eligible patients underwent conventional detection first, followed by AI detection [Supplementary Figure 1, https://links.lww.com/CM9/B248]. For conventional detection, patients underwent conventional colonoscopy by a gastroenterology physician, who was blinded to subsequent video analysis. Patients with unqualified colonoscopy (no cecal intubation, Boston bowel preparation scale score <6, or withdrawal time from cecum to anus [excluding biopsy time] <6 min) or impaired colonoscopy video were excluded. Subsequently, videos of conventional colonoscopy were used for AI detection [Supplementary Figure 2, https://links.lww.com/CM9/B248]. Finally, five assessors from the First Affiliated Hospital, College of Medicine, Zhejiang University reviewed AI-analyzed videos and ruled out false-positive diagnoses. Details of our AI system are shown in Supplementary Materials, https://links.lww.com/CM9/B248. The primary outcome is polyp detection rate (PDR), which is the number of positive patients (those with ≥1 polyp detected during colonoscopy) divided by the total number of patients. Secondary outcomes include polyps per colonoscopy (PPC), calculated as the number of polyps detected during colonoscopy divided by the number of colonoscopies, and polyps per colonoscopy-plus (PPC-Plus), calculated as the number of non-first polyps (polyps detected after the first polyp during colonoscopy) divided by the number of colonoscopies. In addition, we analyzed the characteristics of detected polyps, including location, size, and morphology. All results were analyzed using SPSS 20.0 software (IBM Inc., Chicago, IL, USA). The McNemar test was used to compare the PDR of AI detection and conventional detection, and a comparison between groups was performed using the χ2 test. The PPC and PPC-Plus were compared between conventional detection and AI detection using the Wilcoxon signed-rank test. A P-value <0.05 was considered statistically significant. A total of 764 patients who underwent colonoscopy at Ningbo Hospital of Zhejiang University from May to August 2018 were included in the final analysis. Most (90.4%, 691/764) patients were asymptomatic and underwent colonoscopy for screening or an annual medical examination (Patient characteristics are shown in Supplementary Table 1, https://links.lww.com/CM9/B248). A total of 554 polyps were found in 271 patients by conventional colonoscopy, and 813 polyps were found in 348 patients by AI system. For the primary outcome, the PDR of AI detection was statistically significantly higher than that of conventional detection (45.5% vs. 35.5%, P < 0.001). In addition to the first-found polyps, there were 283 non-first polyps found in conventional detection and 465 in AI detection. For secondary outcomes, PPC (1.1 vs. 0.7, P < 0.001) and PPC-Plus (0.6 vs. 0.4, P < 0.001) were significantly higher in AI detection than in conventional detection [Table 1]. Table 1 - Primary and secondary outcomes for conventional detection and AI detection. Outcomes Conventional detection AI detection Statistical values P values Positive patients∗, n 271 348 Total polyps detected, n 554 813 Non-first polyps† detected, n 283 465 PDR 35.5% 45.5% 16.101 <0.001 PPC 0.7 1.1 −10.957 <0.001 PPC-Plus 0.4 0.6 −8.530 <0.001 ∗Positive patients represent patients with ≥1 polyp detected during conventional detection or AI detection.†Non-first polyps represent polyps detected after the first polyp during colonoscopy. AI: Artificial intelligence; PDR: Polyp detection rate; PPC: Polyps per colonoscopy; PPC-Plus: Polyps per colonoscopy-plus. Further comparing the polyp characteristics of AI and conventional detection, we found that the polyp location distribution was similar, while AI-detected polyps were more likely to be diminutive and flat than conventionally detected polyps [Supplementary Table 2, https://links.lww.com/CM9/B248]. Subanalysis found that the PDR of senior operators was relatively higher than that of junior operators [Supplementary Table 3, https://links.lww.com/CM9/B248]. More critically, the PDR of AI detection was significantly higher than that of conventional detection by both junior (43.9% vs. 34.2%, P < 0.001) and senior operators (47.6% vs. 37.0%, P < 0.001). In this study, we found that more positive patients and additional polyps were detected with AI detection than with conventional colonoscopy, providing strong support for the real-time application of the AI-assisted system. In an early study, Karkanis et al[4] first developed a computer-aided polyp detection system through static image assessment. Here, the AI-assisted dynamic polyp detection system used in the study was developed by ourselves. We conducted a preliminary comparative study on human polyp detection and AI detection, and demonstrated the promise of AI polyp detection through offline video analysis. Compared with conventional colonoscopy, the AI system had better performance in polyp detection, especially for polyps located in the left colon, diminutive and flat, and non-first polyps that are easily missed in conventional colonoscopy. Nowadays, there are several single-center randomized control trials (RCTs) showing the feasibility and effectiveness of online AI-assisted polyp detection.[5] Compared with these systems, our AI system achieves faster real-time detection speeds of 25.8 ms per frame, which motivates us to go a step further by using our system in multicenter RCTs. There are still some limitations of our study. First, the AI failed to detect the 18 polyps found by conventional colonoscopy, suggesting that the AI system cannot fully cover all the findings by the human eye. But when we put this system into live use, it will play an additional role acting as the colonoscopist's “third eye.” Second, there were some false-positive diagnoses due to protruding folds, sucked mucosa, and residual fluid in AI detection. If the system is used for on-site examinations, colonoscopists can check for suspected polyps in real-time and rule out misdiagnosis. Moreover, we did not obtain the histology of the additionally found polyps by AI because the results were analyzed offline. Finally, since all colonoscopies were performed using an OLYMPUS CV-290SL colonoscope, it may cause some biases. In conclusion, our newly developed AI-assisted system has the potential to improve colorectal polyp detection and to reduce operator dependence, which supports the real-time use of this system in colonoscopy and its potential to improve CRC screening and prevention. Funding This study was supported by grants from the Medical Health Science and Technology Project of the Zhejiang Provincial Health Commission (No. 2020KY125) and the National Natural Science Foundation of China (No. 81800511).