Comparing the performance of machine learning and conventional models for predicting atherosclerotic cardiovascular disease in a general Chinese population
Zihao Fan,Zhi Du,Jinrong Fu,Ying Zhou,Pengyu Zhang,Chuning Shi,Yingxian Sun
DOI: https://doi.org/10.1186/s12911-023-02242-z
2023-07-24
Abstract:Background: Accurately predicting the risk of atherosclerotic cardiovascular disease (ASCVD) is crucial for implementing individualized prevention strategies and improving patient outcomes. Our objective is to develop machine learning (ML)-based models for predicting ASCVD risk in a prospective Chinese population and compare their performance with conventional regression models. Methods: A hybrid dataset consisting of 551 features was used, including 98 demographic, behavioral, and psychological features, 444 Electrocardiograph (ECG) features, and 9 Echocardiography (Echo) features. Seven machine learning (ML)-based models were trained, validated, and tested after selecting the 30 most informative features. We compared the discrimination, calibration, net benefit, and net reclassification improvement (NRI) of the ML models with those of conventional ASCVD risk calculators, such as the Pooled Cohort Equations (PCE) and Prediction for ASCVD Risk in China (China-PAR). Results: The study included 9,609 participants (mean age 53.4 ± 10.4 years, 53.7% female), and during a median follow-up of 4.7 years, 431 (4.5%) participants developed ASCVD. In the testing set, the final ML-based ANN model outperformed PCE, China-PAR, recalibrated PCE, and recalibrated China-PAR in predicting ASCVD. This was demonstrated by the model's higher area under the curve (AUC) of 0.800, compared to 0.777, 0.780, 0.779, and 0.779 for the other models, respectively. Additionally, the model had a lower Hosmer-Lemeshow χ2 of 9.1, compared to 37.3, 67.6, 126.6, and 18.6 for the other models. The net benefit at a threshold of 5% was also higher for the ML-based ANN model at 0.017, compared to 0.016, 0.013, 0.017, and 0.016 for the other models, respectively. Furthermore, the NRI was 0.089 for the ML-based ANN model, while it was 0.355, 0.098, and 0.088 for PCE, China-PAR, and recalibrated PCE, respectively. Conclusions: Compared to conventional regression ASCVD risk calculators, such as PCE and China-PAR, the ANN prediction model may help optimize identification of individuals at heightened cardiovascular risk by flexibly incorporating a wider range of potential predictors. The findings may help guide clinical decision-making and ultimately contribute to ASCVD prevention and management.