Effectiveness, Explainability and Reliability of Machine Meta-Learning Methods for Predicting Mortality in Patients with COVID-19: Results of the Brazilian COVID-19 Registry
Polianna Delfino-Pereira,Cláudio Moisés Valiense De Andrade,Virginia Mara Reis Gomes,Maria Clara Pontello Barbosa Lima,Maira Viana Rego Souza-Silva,Marcelo Carneiro,Karina Paula Medeiros Prado Martins,Thaís Lorenna Souza Sales,Rafael Lima Rodrigues De Carvalho,Magda Carvalho Pires,Lucas Emanuel Ferreira Ramos,Rafael T Silva,Adriana Falangola Benjamin Bezerra,Alexandre Vargas Schwarzbold,Aline Gabrielle Sousa Nunes,Amanda de Oliveira Maurilio,Ana Luiza Bahia Alves Scotton,André Soares de Moura Costa,Andriele Abreu Castro,Bárbara Lopes Farace,Christiane Corrêa Rodrigues Cimini,Cíntia Alcântara De Carvalho,Daniel Vitorio Silveira,Daniela Ponce,Elayne Crestani Pereira,Euler Roberto Fernandes Manenti,Evelin Paola de Almeida Cenci,Fernanda Barbosa Lucas,Fernanda d’Athayde Rodrigues,Fernando Anschau,Fernando Antônio Botoni,Fernando Graça Aranha,Frederico Bartolazzi,Gisele Alsina Nader Bastos,Giovanna Grunewald Vietta,Guilherme Fagundes Nascimento,Helena Carolina Noal,Helena Duani,Heloísa Reniers Vianna,Henrique Cerqueira Guimarães,Isabela Moraes Gomes,Jamille Hemerito Salles Martins Costa,Jessica Rayane Corrêa Silva Da Fonseca,Júlia Di Sabatino Santos Guimarães,Júlia Drumond Parreiras De Morais,Juliana Machado Rugolo,Joanna d’Arc Lyra Batista,Joice Coutinho De Alvarenga,José Miguel Chatkin,Karen Brasil Ruschel,Leila Beltrami Moreira,Leonardo Seixas De Oliveira,Liege Barella Zandona,Lilian Santos Pinheiro,Luanna da Silva Monteiro,Lucas de Deus Sousa,Luciane Kopittke,Luciano de Souza Viana,Luís César De Castro,Luísa Argolo Assis,Luísa Elem Almeida Santos,Maderson Álvares de Souza Cabral,Magda Cesar Raposo,Maiara Anschau Floriani,Maria Angélica Pires Ferreira,Maria Aparecida Camargos Bicalho,Mariana Frizzo De Godoy,Matheus Carvalho Alves Nogueira,Meire Pereira De Figueiredo,Milton Henriques Guimarães Júnior,Monica Aparecida de Paula De Sordi,Natália da Cunha Severino Sampaio,Neimy Ramos De Oliveira,Pedro Ledic Assaf,Raquel Lutkmeier,Reginaldo Aparecido Valacio,Renan Goulart Finger,Rochele Mosmann Menezes,Rufino de Freitas Silva,Saionara Cristina Francisco,Silvana Mangeon Meireles Guimaraes,Silvia Ferreira Araujo,Talita Fischer Oliveira,Tatiana Kurtz,Tatiana Oliveira Fereguetti,Thainara Conceição De Oliveira,Túlio Henrique Oliveira Diniz,Yara Cristina Neves Marques Barbosa Ribeiro,Yuri Carlotto Ramires,Marcos André Gonçalves,Milena Soriano Marcolino,Bruno Barbosa Miranda de Paiva
DOI: https://doi.org/10.21203/rs.3.rs-1164411/v1
2022-01-11
Abstract:Abstract The majority prognostic scores proposed for early assessment of coronavirus disease 19 (COVID-19) patients are bounded by methodological flaws. Our group recently developed a new risk score - ABC 2 SPH - using traditional statistical methods (least absolute shrinkage and selection operator logistic regression - LASSO). In this article, we provide a thorough comparative study between modern machine learning (ML) methods and state-of-the-art statistical methods, represented by ABC 2 SPH, in the task of predicting in-hospital mortality in COVID-19 patients using data upon hospital admission. We overcome methodological and technological issues found in previous similar studies, while exploring a large sample (5,032 patients). Additionally, we take advantage of a large and diverse set of methods and investigate the effectiveness of applying meta-learning, more specifically Stacking, in order to combine the methods' strengths and overcome their limitations. In our experiments, our Stacking solutions improved over previous state-of-the-art by more than 26% in predicting death, achieving 87.1% of AUROC and MacroF1 of 73.9%. We also investigated issues related to the interpretability and reliability of the predictions produced by the most effective ML methods. Finally, we discuss the adequacy of AUROC as an evaluation metric for highly imbalanced and skewed datasets commonly found in health-related problems.