The Brain Tumor Segmentation - Metastases (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI
Ahmed W Moawad,Anastasia Janas,Ujjwal Baid,Divya Ramakrishnan,Rachit Saluja,Nader Ashraf,Leon Jekel,Raisa Amiruddin,Maruf Adewole,Jake Albrecht,Udunna Anazodo,Sanjay Aneja,Syed Muhammad Anwar,Timothy Bergquist,Evan Calabrese,Veronica Chiang,Verena Chung,Gian Marco Marco Conte,Farouk Dako,James Eddy,Ivan Ezhov,Ariana Familiar,Keyvan Farahani,Juan Eugenio Iglesias,Zhifan Jiang,Elaine Johanson,Anahita Fathi Kazerooni,Florian Kofler,Kiril Krantchev,Dominic LaBella,Koen Van Leemput,Hongwei Bran Li,Marius George Linguraru,Katherine E Link,Xinyang Liu,Nazanin Maleki,Zeke Meier,Bjoern H Menze,Harrison Moy,Klara Osenberg,Marie Piraud,Zachary Reitman,Russel Takeshi Shinohara,Nourel Hoda Tahon,Ayman Nada,Yuri S Velichko,Chunhao Wang,Benedikt Wiestler,Walter Wiggins,Umber Shafique,Klara Willms,Arman Avesta,Khaled Bousabarah,Satrajit Chakrabarty,Nicolo Gennaro,Wolfgang Holler,Manpreet Kaur,Pamela LaMontagne,MingDe Lin,Jan Lost,Daniel S Marcus,Ryan Maresca,Sarah Merkaj,Ayaman Nada,Gabriel Cassinelli Pedersen,Marc von Reppert,Aristeidis Sotiras,Oleg Teytelboym,Niklas Tillmans,Malte Westerhoff,Ayda Youssef,Devon Godfrey,Scott Floyd,Andreas Rauschecker,Javier Villanueva-Meyer,Irada Pflüger,Jaeyoung Cho,Martin Bendszus,Gianluca Brugnara,Justin Cramer,Gloria J Guzman Perez-Carillo,Derek R Johnson,Anthony Kam,Benjamin Yin Ming Kwan,Lillian Lai,Neil U Lall,Fatima Memon,Satya Narayana Patro,Bojan Petrovic,Tiffany Y So,Gerard Thompson,Lei Wu,E Brooke Schrickel,Anu Bansal,Frederik Barkhof,Cristina Besada,Sammy Chu,Jason Druzgal,Alexandru Dusoi,Luciano Farage,Fabricio Feltrin,Amy Fong,Steve H Fung,R Ian Gray,Ichiro Ikuta,Michael Iv,Alida A Postma,Amit Mahajan,David Joyner,Chase Krumpelman,Laurent Letourneau-Guillon,Christie M Lincoln,Mate E Maros,Elka Miller,Fanny Morón,Esther A Nimchinsky,Ozkan Ozsarlak,Uresh Patel,Saurabh Rohatgi,Atin Saha,Anousheh Sayah,Eric D Schwartz,Robert Shih,Mark S Shiroishi,Juan E Small,Manoj Tanwar,Jewels Valerie,Brent D Weinberg,Matthew L White,Robert Young,Vahe M Zohrabian,Aynur Azizova,Melanie Maria Theresa Brüßeler,Pascal Fehringer,Mohanad Ghonim,Mohamed Ghonim,Athanasios Gkampenis,Abdullah Okar,Luca Pasquini,Yasaman Sharifi,Gagandeep Singh,Nico Sollmann,Theodora Soumala,Mahsa Taherzadeh,Nikolay Yordanov,Philipp Vollmuth,Martha Foltyn-Dumitru,Ajay Malhotra,Aly H Abayazeed,Francesco Dellepiane,Philipp Lohmann,Víctor M Pérez-García,Hesham Elhalawani,Sanaria Al-Rubaiey,Rui Duarte Armindo,Kholod Ashraf,Moamen M Asla,Mohamed Badawy,Jeroen Bisschop,Nima Broomand Lomer,Jan Bukatz,Jim Chen,Petra Cimflova,Felix Corr,Alexis Crawley,Lisa Deptula,Tasneem Elakhdar,Islam H Shawali,Shahriar Faghani,Alexandra Frick,Vaibhav Gulati,Muhammad Ammar Haider,Fátima Hierro,Rasmus Holmboe Dahl,Sarah Maria Jacobs,Kuang-Chun Jim Hsieh,Sedat G Kandemirli,Katharina Kersting,Laura Kida,Sofia Kollia,Ioannis Koukoulithras,Xiao Li,Ahmed Abouelatta,Aya Mansour,Ruxandra-Catrinel Maria-Zamfirescu,Marcela Marsiglia,Yohana Sarahi Mateo-Camacho,Mark McArthur,Olivia McDonnell,Maire McHugh,Mana Moassefi,Samah Mostafa Morsi,Alexander Muntenu,Khanak K Nandolia,Syed Raza Naqvi,Yalda Nikanpour,Mostafa Alnoury,Abdullah Mohamed Aly Nouh,Francesca Pappafava,Markand D Patel,Samantha Petrucci,Eric Rawie,Scott Raymond,Borna Roohani,Sadeq Sabouhi,Laura M Sanchez-Garcia,Zoe Shaked,Pokhraj P Suthar,Talissa Altes,Edvin Isufi,Yaseen Dhermesh,Jaime Gass,Jonathan Thacker,Abdul Rahman Tarabishy,Benjamin Turner,Sebastiano Vacca,George K Vilanilam,Daniel Warren,David Weiss,Fikadu Worede,Sara Yousry,Wondwossen Lerebo,Alejandro Aristizabal,Alexandros Karargyris,Hasan Kassem,Sarthak Pati,Micah Sheller,Spyridon Bakas,Jeffrey D Rudie,Mariam Aboian
2024-06-17
Abstract:The translation of AI-generated brain metastases (BM) segmentation into clinical practice relies heavily on diverse, high-quality annotated medical imaging datasets. The BraTS-METS 2023 challenge has gained momentum for testing and benchmarking algorithms using rigorously annotated internationally compiled real-world datasets. This study presents the results of the segmentation challenge and characterizes the challenging cases that impacted the performance of the winning algorithms. Untreated brain metastases on standard anatomic MRI sequences (T1, T2, FLAIR, T1PG) from eight contributed international datasets were annotated in stepwise method: published UNET algorithms, student, neuroradiologist, final approver neuroradiologist. Segmentations were ranked based on lesion-wise Dice and Hausdorff distance (HD95) scores. False positives (FP) and false negatives (FN) were rigorously penalized, receiving a score of 0 for Dice and a fixed penalty of 374 for HD95. The mean scores for the teams were calculated. Eight datasets comprising 1303 studies were annotated, with 402 studies (3076 lesions) released on Synapse as publicly available datasets to challenge competitors. Additionally, 31 studies (139 lesions) were held out for validation, and 59 studies (218 lesions) were used for testing. Segmentation accuracy was measured as rank across subjects, with the winning team achieving a LesionWise mean score of 7.9. The Dice score for the winning team was 0.65 ± 0.25. Common errors among the leading teams included false negatives for small lesions and misregistration of masks in space. The Dice scores and lesion detection rates of all algorithms diminished with decreasing tumor size, particularly for tumors smaller than 100 mm3. In conclusion, algorithms for BM segmentation require further refinement to balance high sensitivity in lesion detection with the minimization of false positives and negatives. The BraTS-METS 2023 challenge successfully curated well-annotated, diverse datasets and identified common errors, facilitating the translation of BM segmentation across varied clinical environments and providing personalized volumetric reports to patients undergoing BM treatment.