Automatic Captioning of Early Gastric Cancer via Magnification Endoscopy with Narrow Band Imaging

Lixin Gong,Min Wang,Lei Shu,Jie He,Bin Qin,Jiacheng Xu,Wei Su,Di Dong,Hao Hu,Jie Tian,Pinghong Zhou
DOI: https://doi.org/10.1016/j.gie.2022.07.019
IF: 10.396
2022-07-31
Gastrointestinal Endoscopy
Abstract:Background and Aims Early gastric cancer (EGC) detection rate is unsatisfactory, and mastering the diagnostic skills of magnifying endoscopy with narrow-band imaging (ME-NBI) requires rich expertise and experience. We aimed to develop an EGC captioning model (EGCCap) to automatically describe the visual characteristics of ME-NBI images for endoscopists. Methods A total of 1886 ME-NBI images from 294 cases were enrolled from multi-centers, and corresponding 5658 text data was designed following the simple EGC diagnostic algorithm. An EGCCap was developed using the multi-scale meshed-memory transformer. We conducted comprehensive evaluations for EGCCap including the quantitative and quality performance, generalization, robustness, interpretability, and assistant value analyses. The commonly used metrics including BLEUs, CIDEr, METEOR, ROUGE, SPICE, accuracy, sensitivity, and specificity were used. Two-sided statistical tests were conducted, and statistical significance was determined when P<0.05. Results EGCCap acquired satisfying captioning performance by outputting correctly and coherently clinically meaningful sentences in the internal test cohort (BLEU1 = 52.434, CIDEr = 36.734, METEOR = 27.823, ROUGE = 49.949, SPICE = 35.548), and it maintained over 80% performance when applied to other centers or corrupted data. The diagnostic ability of endoscopists improved with the assistance of EGCCap, which was especially significant (P<0.05) for junior endoscopists. Endoscopists gave EGCCap an average remarkable score of 7.182, showing acceptance of EGCCap. Conclusion EGCCap exhibited promising captioning performance and was proved with satisfying generalization, robustness, and interpretability. Our study showed potential value in aiding and improving the diagnosis of EGC and facilitating the development of automated reporting in the future.
gastroenterology & hepatology
What problem does this paper attempt to address?