An AI deep learning algorithm for detecting pulmonary nodules on ultra-low-dose CT in an emergency setting: a reader study
Inge A H van den Berk,Colin Jacobs,Maadrika M N P Kanglie,Onno M Mets,Miranda Snoeren,Alexander D Montauban van Swijndregt,Elisabeth M Taal,Tjitske S R van Engelen,Jan M Prins,Shandra Bipat,Patrick M M Bossuyt,Jaap Stoker,OPTIMACT study group,Jouke Annema,Ludo F M Beenen,Dominique Bekebrede-Kaufman,Brenda Elzer,Betty Frankemölle,Maarten Groenink,Erwin Hoolwerf,Dorine Hulzebosch,Saskia Kolkman,Nick H J Lobe,Peter A Leenhouts,Onno Mets,Melanie A Monraats,Jan Luitse,Saskia Middeldorp,Alexander Montauban van Swijndregt,Jacqueline Otker,Adrienne van Randen,Milan L Ridderikhof,Johannes A Romijn,Maeke J Scheerder,Antoinet J N Schoonderwoerd,Laura J Schijf,Frank F Smithuis,Geert J Streekstra,Elizabeth M Taal,Glenn de Vries,Maaike J A Vogel,Ibtisam Yahya
DOI: https://doi.org/10.1186/s41747-024-00518-1
2024-11-20
Abstract:Background: To retrospectively assess the added value of an artificial intelligence (AI) algorithm for detecting pulmonary nodules on ultra-low-dose computed tomography (ULDCT) performed at the emergency department (ED). Methods: In the OPTIMACT trial, 870 patients with suspected nontraumatic pulmonary disease underwent ULDCT. The ED radiologist prospectively read the examinations and reported incidental pulmonary nodules requiring follow-up. All ULDCTs were processed post hoc using an AI deep learning software marking pulmonary nodules ≥ 6 mm. Three chest radiologists independently reviewed the subset of ULDCTs with either prospectively detected incidental nodules in 35/870 patients or AI marks in 458/870 patients; findings scored as nodules by at least two chest radiologists were used as true positive reference standard. Proportions of true and false positives were compared. Results: During the OPTIMACT study, 59 incidental pulmonary nodules requiring follow-up were prospectively reported. In the current analysis, 18/59 (30.5%) nodules were scored as true positive while 104/1,862 (5.6%) AI marks in 84/870 patients (9.7%) were scored as true positive. Overall, 5.8 times more (104 versus 18) true positive pulmonary nodules were detected with the use of AI, at the expense of 42.9 times more (1,758 versus 41) false positives. There was a median number of 1 (IQR: 0-2) AI mark per ULDCT. Conclusion: The use of AI on ULDCT in patients suspected of pulmonary disease in an emergency setting results in the detection of many more incidental pulmonary nodules requiring follow-up (5.8×) with a high trade-off in terms of false positives (42.9×). Relevance statement: AI aids in the detection of incidental pulmonary nodules that require follow-up at chest-CT, aiding early pulmonary cancer detection but also results in an increase of false positive results that are mainly clustered in patients with major abnormalities. Trial registration: The OPTIMACT trial was registered on 6 December 2016 in the National Trial Register (number NTR6163) (onderzoekmetmensen.nl). Key points: An AI deep learning algorithm was tested on 870 ULDCT examinations acquired in the ED. AI detected 5.8 times more pulmonary nodules requiring follow-up (true positives). AI resulted in the detection of 42.9 times more false positive results, clustered in patients with major abnormalities. AI in the ED setting may aid in early pulmonary cancer detection with a high trade-off in terms of false positives.