Voting of predictive models for clinical outcomes: consensus of algorithms for the early prediction of sepsis from clinical data and an analysis of the PhysioNet/Computing in Cardiology Challenge 2019

Matthew A. Reyna,Gari D. Clifford
DOI: https://doi.org/10.48550/arXiv.2012.11013
2020-12-21
Abstract:Although there has been significant research in boosting of weak learners, there has been little work in the field of boosting from strong learners. This latter paradigm is a form of weighted voting with learned weights. In this work, we consider the problem of constructing an ensemble algorithm from 70 individual algorithms for the early prediction of sepsis from clinical data. We find that this ensemble algorithm outperforms separate algorithms, especially on a hidden test set on which most algorithms failed to generalize.
Machine Learning
What problem does this paper attempt to address?
### What problem does this paper attempt to solve? This paper aims to solve the problem of early prediction of sepsis from clinical data. Specifically, the authors explored how to improve the accuracy of early prediction of sepsis by integrating multiple independent algorithms, especially the generalization ability on the hidden test set. #### Background and Motivation 1. **Importance of Clinical Problems**: Sepsis is a serious clinical condition, and early prediction is crucial for timely treatment. However, the performance of existing single algorithms varies greatly in different hospital systems, especially with insufficient generalization ability on unseen data sets. 2. **Diversity and Ensemble Learning**: Although existing research has shown that simple voting models can outperform single models in some clinical tasks (such as electrocardiogram classification), research on how to use diverse algorithms for ensemble learning is still limited. In particular, how to conduct weighted voting among strong learners to improve prediction performance is an area that has not been fully explored. #### Research Objectives 1. **Construct an Ensemble Algorithm**: The authors attempted to construct an ensemble algorithm from 70 different algorithms designed for the PhysioNet/Computing in Cardiology Challenge 2019 for the early prediction of sepsis. 2. **Evaluate Generalization Ability**: Pay special attention to the performance of the ensemble algorithm on the completely hidden test set to evaluate its generalization ability. The hidden test set includes data from different hospital systems, which helps to verify whether the model can adapt to new, unseen data. 3. **Improve Prediction Performance**: Through the ensemble learning method, especially weighted voting based on the voting mechanism, the authors hope to find a method that can surpass single algorithms, thereby improving the accuracy of early sepsis prediction. #### Main Contributions - **Proposed a Voting - Based Ensemble Algorithm**: This algorithm can significantly outperform single algorithms on test data from different hospital systems, especially those hospital systems not included in the training set. - **Analyzed the Impact of Algorithm Diversity**: By comparing the code similarity and prediction results of different algorithms, the authors revealed the high consistency among high - scoring algorithms and the impact of this consistency on the performance of the ensemble model. - **Emphasized the Importance of Generalization Ability**: The research results show that generalization ability on unseen data sets is a key indicator for evaluating model performance, and the ensemble learning method can effectively improve this ability. In general, this paper shows how to use diverse algorithms to improve the accuracy and generalization ability of early sepsis prediction by constructing and evaluating a voting - based ensemble algorithm.