Effect of aflatoxin B1 on net synthesis of albumin, fibrinogen, and alpha-1-acid glycoprotein by the isolated perfused rat liver.

D. W. John,L. L. Miller

IF: 6.1

Biochemical Pharmacology

Abstract:

What problem does this paper attempt to address?

Interpretable (not just posthoc-explainable) heterogeneous survivor bias-corrected treatment effects for assignment of postdischarge interventions to prevent readmissions

Hongjing Xia,Joshua C. Chang,Sarah Nowak,Sonya Mahajan,Rohit Mahajan,Ted L. Chang,Carson C. Chow

2023-08-04

Abstract:We used survival analysis to quantify the impact of postdischarge evaluation and management (E/M) services in preventing hospital readmission or death. Our approach avoids a specific pitfall of applying machine learning to this problem, which is an inflated estimate of the effect of interventions, due to survivors bias -- where the magnitude of inflation may be conditional on heterogeneous confounders in the population. This bias arises simply because in order to receive an intervention after discharge, a person must not have been readmitted in the intervening period. After deriving an expression for this phantom effect, we controlled for this and other biases within an inherently interpretable Bayesian survival framework. We identified case management services as being the most impactful for reducing readmissions overall.

Methodology,Machine Learning,Quantitative Methods
Small samples-oriented intrinsically explainable machine learning using Variational Bayesian Logistic Regression: An intensive care unit readmission prediction case for liver transplantation patients

Jinping Liu,Xiaoqiang Wu,Yongming Xie,Zhaohui Tang,Yongfang Xie,Subo Gong

DOI: https://doi.org/10.1016/j.eswa.2023.121138

IF: 8.5

2023-08-21

Expert Systems with Applications

Abstract:Recently, machine learning has revolutionized medical diagnosis and prediction, rivaling human experts in accuracy. However, the limited interpretability poses challenges in meeting stringent medical decision standards. To facilitate interpretable predictive inference , this article proposes a small samples-oriented intrinsically explainable machine learning model, an Optimal Variational Bayesian Logistic Regression (OVBLR) model with a salient feature estimation strategy (OVBLR-SFE). OVBLR-SFE sequentially learns from limited labeled samples, mining disease-related risk factors to provide intrinsic interpretability to its predictive inference automatically. The model achieves this by reasonably approximating the posterior probability through the proposed OVBLR model to estimate the feature weight and influential effect on model outputs to facilitate salient feature selection. The effectiveness and superiority of OVBLR-SFE are demonstrated through extensive experiments on four benchmark medical datasets from the UCI repository, including the Bone Marrow Transplant: Children Data Set, Breast Cancer Wisconsin (Original) Data Set, SPECT Heart Data Set, and Heart Disease Data Set, as well as a real-world case of intensive care unit readmission prediction. Experimental results reveal that the OVBLR-SFE can achieve high accuracies of 92.02%, 97.21%, 86.71%, and 84.47%, respectively, with an average classification accuracy of 90.10% stably on the four UCI datasets. Especially, OVBLR-SFE can accurately infer significant features that align with that obtained by the post-hoc interpretation approaches: SHAP and AcME, demonstrating remarkable interpretability. In the real application, the proposed method achieved a readmission prediction accuracy of 88.11% in predicting the readmission probabilities of liver transplantation patients. Notably, the inferred salient features, such as total Intensive Care Unit length of stay (ICUTime), N-terminal pro-brain natriuretic peptide(NTproBNP), Operating time of surgery(OperationTime_h), and intraoperative red blood cell transfusion (RBCT), emerges as crucial in readmission prediction, aligning with the assessments of clinical experts. The source code of this study can be accessed at https://github.com/xqw42/OVBLR-SFE .

computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science
Hospital Readmission Prediction - Applying Hierarchical Sparsity Norms for Interpretable Models

Jialiang Jiang,Sharon Hewner,Varun Chandola

DOI: https://doi.org/10.48550/arXiv.1804.01188

IF: 5.414

2018-04-03

Machine Learning

Abstract:Hospital readmissions have become one of the key measures of healthcare quality. Preventable readmissions have been identified as one of the primary targets for reducing costs and improving healthcare delivery. However, most data driven studies for understanding readmissions have produced black box classification and predictive models with moderate performance, which precludes them from being used effectively within the decision support systems in the hospitals. In this paper we present an application of structured sparsity-inducing norms for predicting readmission risk for patients based on their disease history and demographics. Most existing studies have focused on hospital utilization, test results, etc., to assign a readmission label to each episode of hospitalization. However, we focus on assigning a readmission risk label to a patient based on their disease history. Our emphasis is on interpreting the models to improve the understanding of the readmission problem. To achieve this, we exploit the domain induced hierarchical structure available for the disease codes which are the features for the classification algorithm. We use a tree based sparsity-inducing regularization strategy that explicitly uses the domain hierarchy. The resulting model not only outperforms standard regularization procedures but is also highly sparse and interpretable. We analyze the model and identify several significant factors that have an effect on readmission risk. Some of these factors conform to existing beliefs, e.g., impact of surgical complications and infections during hospital stay. Other factors, such as the impact of mental disorder and substance abuse on readmission, provide empirical evidence for several pre-existing but unverified hypotheses. The analysis also reveals previously undiscovered connections such as the influence of socioeconomic factors like lack of housing and malnutrition.
The Price of Explainability in Machine Learning Models for 100-Day Readmission Prediction in Heart Failure: Retrospective, Comparative, Machine Learning Study

Amira Soliman,Björn Agvall,Kobra Etminani,Omar Hamed,Markus Lingman

DOI: https://doi.org/10.2196/46934

IF: 7.076

2023-10-27

Journal of Medical Internet Research

Abstract:Background Sensitive and interpretable machine learning (ML) models can provide valuable assistance to clinicians in managing patients with heart failure (HF) at discharge by identifying individual factors associated with a high risk of readmission. In this cohort study, we delve into the factors driving the potential utility of classification models as decision support tools for predicting readmissions in patients with HF. Objective The primary objective of this study is to assess the trade-off between using deep learning (DL) and traditional ML models to identify the risk of 100-day readmissions in patients with HF. Additionally, the study aims to provide explanations for the model predictions by highlighting important features both on a global scale across the patient cohort and on a local level for individual patients. Methods The retrospective data for this study were obtained from the Regional Health Care Information Platform in Region Halland, Sweden. The study cohort consisted of patients diagnosed with HF who were over 40 years old and had been hospitalized at least once between 2017 and 2019. Data analysis encompassed the period from January 1, 2017, to December 31, 2019. Two ML models were developed and validated to predict 100-day readmissions, with a focus on the explainability of the model’s decisions. These models were built based on decision trees and recurrent neural architecture. Model explainability was obtained using an ML explainer. The predictive performance of these models was compared against 2 risk assessment tools using multiple performance metrics. Results The retrospective data set included a total of 15,612 admissions, and within these admissions, readmission occurred in 5597 cases, representing a readmission rate of 35.85%. It is noteworthy that a traditional and explainable model, informed by clinical knowledge, exhibited performance comparable to the DL model and surpassed conventional scoring methods in predicting readmission among patients with HF. The evaluation of predictive model performance was based on commonly used metrics, with an area under the precision-recall curve of 66% for the deep model and 68% for the traditional model on the holdout data set. Importantly, the explanations provided by the traditional model offer actionable insights that have the potential to enhance care planning. Conclusions This study found that a widely used deep prediction model did not outperform an explainable ML model when predicting readmissions among patients with HF. The results suggest that model transparency does not necessarily compromise performance, which could facilitate the clinical adoption of such models.

health care sciences & services,medical informatics
Interpretable machine learning models for hospital readmission prediction: a two-step extracted regression tree approach

Xiaoquan Gao,Sabriya Alam,Pengyi Shi,Franklin Dexter,Nan Kong

DOI: https://doi.org/10.1186/s12911-023-02193-5

IF: 3.298

2023-06-06

BMC Medical Informatics and Decision Making

Abstract:Advanced machine learning models have received wide attention in assisting medical decision making due to the greater accuracy they can achieve. However, their limited interpretability imposes barriers for practitioners to adopt them. Recent advancements in interpretable machine learning tools allow us to look inside the black box of advanced prediction methods to extract interpretable models while maintaining similar prediction accuracy, but few studies have investigated the specific hospital readmission prediction problem with this spirit.

medical informatics
Comparative analysis of explainable machine learning prediction models for hospital mortality

Eline Stenwig,Giampiero Salvi,Pierluigi Salvo Rossi,Nils Kristian Skjærvold

DOI: https://doi.org/10.1186/s12874-022-01540-w

2022-02-27

BMC Medical Research Methodology

Abstract:Abstract Background Machine learning (ML) holds the promise of becoming an essential tool for utilising the increasing amount of clinical data available for analysis and clinical decision support. However, the lack of trust in the models has limited the acceptance of this technology in healthcare. This mistrust is often credited to the shortage of model explainability and interpretability, where the relationship between the input and output of the models is unclear. Improving trust requires the development of more transparent ML methods. Methods In this paper, we use the publicly available eICU database to construct a number of ML models before examining their internal behaviour with SHapley Additive exPlanations (SHAP) values. Our four models predicted hospital mortality in ICU patients using a selection of the same features used to calculate the APACHE IV score and were based on random forest, logistic regression, naive Bayes, and adaptive boosting algorithms. Results The results showed the models had similar discriminative abilities and mostly agreed on feature importance while calibration and impact of individual features differed considerably and did in multiple cases not correspond to common medical theory. Conclusions We already know that ML models treat data differently depending on the underlying algorithm. Our comparative analysis visualises implications of these differences and their importance in a healthcare setting. SHAP value analysis is a promising method for incorporating explainability in model development and usage and might yield better and more trustworthy ML models in the future.

health care sciences & services
Predicting 30-day Hospital Readmission with Publicly Available Administrative Database. A Conditional Logistic Regression Modeling Approach

K Zhu,Z Lou,J Zhou,N Ballester,N Kong,P Parikh

DOI: https://doi.org/10.3414/ME14-02-0017

Abstract:Introduction: This article is part of the Focus Theme of Methods of Information in Medicine on "Big Data and Analytics in Healthcare". Background: Hospital readmissions raise healthcare costs and cause significant distress to providers and patients. It is, therefore, of great interest to healthcare organizations to predict what patients are at risk to be readmitted to their hospitals. However, current logistic regression based risk prediction models have limited prediction power when applied to hospital administrative data. Meanwhile, although decision trees and random forests have been applied, they tend to be too complex to understand among the hospital practitioners. Objectives: Explore the use of conditional logistic regression to increase the prediction accuracy. Methods: We analyzed an HCUP statewide inpatient discharge record dataset, which includes patient demographics, clinical and care utilization data from California. We extracted records of heart failure Medicare beneficiaries who had inpatient experience during an 11-month period. We corrected the data imbalance issue with under-sampling. In our study, we first applied standard logistic regression and decision tree to obtain influential variables and derive practically meaning decision rules. We then stratified the original data set accordingly and applied logistic regression on each data stratum. We further explored the effect of interacting variables in the logistic regression modeling. We conducted cross validation to assess the overall prediction performance of conditional logistic regression (CLR) and compared it with standard classification models. Results: The developed CLR models outperformed several standard classification models (e.g., straightforward logistic regression, stepwise logistic regression, random forest, support vector machine). For example, the best CLR model improved the classification accuracy by nearly 20% over the straightforward logistic regression model. Furthermore, the developed CLR models tend to achieve better sensitivity of more than 10% over the standard classification models, which can be translated to correct labeling of additional 400 - 500 readmissions for heart failure patients in the state of California over a year. Lastly, several key predictor identified from the HCUP data include the disposition location from discharge, the number of chronic conditions, and the number of acute procedures. Conclusions: It would be beneficial to apply simple decision rules obtained from the decision tree in an ad-hoc manner to guide the cohort stratification. It could be potentially beneficial to explore the effect of pairwise interactions between influential predictors when building the logistic regression models for different data strata. Judicious use of the ad-hoc CLR models developed offers insights into future development of prediction models for hospital readmissions, which can lead to better intuition in identifying high-risk patients and developing effective post-discharge care strategies. Lastly, this paper is expected to raise the awareness of collecting data on additional markers and developing necessary database infrastructure for larger-scale exploratory studies on readmission risk prediction.
SCOPE: predicting future diagnoses in office visits using electronic health records

Pritam Mukherjee,Marie Humbert-Droz,Jonathan H Chen,Olivier Gevaert

DOI: https://doi.org/10.1038/s41598-023-38257-9

2023-07-07

Abstract:We propose an interpretable and scalable model to predict likely diagnoses at an encounter based on past diagnoses and lab results. This model is intended to aid physicians in their interaction with the electronic health records (EHR). To accomplish this, we retrospectively collected and de-identified EHR data of 2,701,522 patients at Stanford Healthcare over a time period from January 2008 to December 2016. A population-based sample of patients comprising 524,198 individuals (44% M, 56% F) with multiple encounters with at least one frequently occurring diagnosis codes were chosen. A calibrated model was developed to predict ICD-10 diagnosis codes at an encounter based on the past diagnoses and lab results, using a binary relevance based multi-label modeling strategy. Logistic regression and random forests were tested as the base classifier, and several time windows were tested for aggregating the past diagnoses and labs. This modeling approach was compared to a recurrent neural network based deep learning method. The best model used random forest as the base classifier and integrated demographic features, diagnosis codes, and lab results. The best model was calibrated and its performance was comparable or better than existing methods in terms of various metrics, including a median AUROC of 0.904 (IQR [0.838, 0.954]) over 583 diseases. When predicting the first occurrence of a disease label for a patient, the median AUROC with the best model was 0.796 (IQR [0.737, 0.868]). Our modeling approach performed comparably as the tested deep learning method, outperforming it in terms of AUROC (p < 0.001) but underperforming in terms of AUPRC (p < 0.001). Interpreting the model showed that the model uses meaningful features and highlights many interesting associations among diagnoses and lab results. We conclude that the multi-label model performs comparably with RNN based deep learning model while offering simplicity and potentially superior interpretability. While the model was trained and validated on data obtained from a single institution, its simplicity, interpretability and performance makes it a promising candidate for deployment.
Leveraging deep survival models to predict quality of care risk in diverse hospital readmissions

Nhat Quang Tran,Gautam Goel,Nirmala Pudota,Michael Suesserman,John Helms,Daniel Lasaga,Dan Olson,Edward Bowen,Sanmitra Bhattacharya

DOI: https://doi.org/10.1038/s41598-023-37477-3

2023-06-28

Abstract:Hospital readmissions rate is reportedly high and has caused huge financial burden on health care systems in many countries. It is viewed as an important indicator of health care providers' quality of care. We examine the use of machine learning-based survival analysis to assess quality of care risk in hospital readmissions. This study applies various survival models to explore the risk of hospital readmissions given patient demographics and their respective hospital discharges extracted from a health care claims dataset. We explore advanced feature representation techniques such as BioBERT and Node2Vec to encode high-dimensional diagnosis code features. To our knowledge, this study is the first to apply deep-learning based survival-analysis models for predicting hospital readmission risk agnostic of specific medical conditions and a fixed window for readmission. We found that modeling the time from discharge date to readmission date as a Weibull distribution as in the SparseDeepWeiSurv model yields the best discriminative power and calibration. In addition, embedding representations of the diagnosis codes do not contribute to improvement in model performance. We find dependency of each model's performance on the time point at which it is evaluated. This time dependency of the models' performance on the health care claims data may necessitate a different choice of model in quality of care issue detection at different points in time. We show the effectiveness of deep-learning based survival-analysis models in estimating the quality of care risk in hospital readmissions.
Explainable machine learning for predicting conversion to neurological disease: Results from 52,939 medical records

Christina Felix,Joshua D Johnston,Kelsey Owen,Emil Shirima,Sidney R Hinds,Kenneth D Mandl,Alex Milinovich,Jay L Alberts

DOI: https://doi.org/10.1177/20552076241249286

2024-01-01

Digital Health

Abstract:Objective This study assesses the application of interpretable machine learning modeling using electronic medical record data for the prediction of conversion to neurological disease. Methods A retrospective dataset of Cleveland Clinic patients diagnosed with Alzheimer's disease, amyotrophic lateral sclerosis, multiple sclerosis, or Parkinson's disease, and matched controls based on age, sex, race, and ethnicity was compiled. Individualized risk prediction models were created using eXtreme Gradient Boosting for each neurological disease at four timepoints in patient history. The prediction models were assessed for transparency and fairness. Results At timepoints 0-months, 12-months, 24-months, and 60-months prior to diagnosis, Alzheimer’s disease models achieved the area under the receiver operating characteristic curve on a holdout test dataset of 0.794, 0.742, 0.709, and 0.645; amyotrophic lateral sclerosis of 0.883, 0.710, 0.658, and 0.620; multiple sclerosis of 0.922, 0.877, 0.849, and 0.781; and Parkinson’s disease of 0.809, 0.738, 0.700, and 0.651, respectively. Conclusions The results demonstrate that electronic medical records contain latent information that can be used for risk stratification for neurological disorders. In particular, patient-reported outcomes, sleep assessments, falls data, additional disease diagnoses, and longitudinal changes in patient health, such as weight change, are important predictors.

public, environmental & occupational health,health care sciences & services,medical informatics,health policy & services
Explainable Prediction of Adverse Outcomes Using Clinical Notes

Justin R. Lovelace,Nathan C. Hurley,Adrian D. Haimovich,Bobak J. Mortazavi

DOI: https://doi.org/10.48550/arXiv.1910.14095

2019-11-13

Abstract:Clinical notes contain a large amount of clinically valuable information that is ignored in many clinical decision support systems due to the difficulty that comes with mining that information. Recent work has found success leveraging deep learning models for the prediction of clinical outcomes using clinical notes. However, these models fail to provide clinically relevant and interpretable information that clinicians can utilize for informed clinical care. In this work, we augment a popular convolutional model with an attention mechanism and apply it to unstructured clinical notes for the prediction of ICU readmission and mortality. We find that the addition of the attention mechanism leads to competitive performance while allowing for the straightforward interpretation of predictions. We develop clear visualizations to present important spans of text for both individual predictions and high-risk cohorts. We then conduct a qualitative analysis and demonstrate that our model is consistently attending to clinically meaningful portions of the narrative for all of the outcomes that we explore.

Machine Learning
Interpretable Neural Networks for Predicting Mortality Risk using Multi-modal Electronic Health Records

Alvaro E. Ulloa Cerna,Marios Pattichis,David P. vanMaanen,Linyuan Jing,Aalpen A. Patel,Joshua V. Stough,Christopher M. Haggerty,Brandon K. Fornwalt

DOI: https://doi.org/10.48550/arXiv.1901.08125

2019-01-24

Abstract:We present an interpretable neural network for predicting an important clinical outcome (1-year mortality) from multi-modal Electronic Health Record (EHR) data. Our approach builds on prior multi-modal machine learning models by now enabling visualization of how individual factors contribute to the overall outcome risk, assuming other factors remain constant, which was previously impossible. We demonstrate the value of this approach using a large multi-modal clinical dataset including both EHR data and 31,278 echocardiographic videos of the heart from 26,793 patients. We generated separate models for (i) clinical data only (CD) (e.g. age, sex, diagnoses and laboratory values), (ii) numeric variables derived from the videos, which we call echocardiography-derived measures (EDM), and (iii) CD+EDM+raw videos (pixel data). The interpretable multi-modal model maintained performance compared to non-interpretable models (Random Forest, XGBoost), and also performed significantly better than a model using a single modality (average AUC=0.82). Clinically relevant insights and multi-modal variable importance rankings were also facilitated by the new model, which have previously been impossible.

Machine Learning
Explainable artificial intelligence model for mortality risk prediction in the intensive care unit: a derivation and validation study

Chang Hu,Chao Gao,Tianlong Li,Chang Liu,Zhiyong Peng

DOI: https://doi.org/10.1093/postmj/qgad144

2024-01-19

Postgraduate Medical Journal

Abstract:Abstract Background The lack of transparency is a prevalent issue among the current machine-learning (ML) algorithms utilized for predicting mortality risk. Herein, we aimed to improve transparency by utilizing the latest ML explicable technology, SHapley Additive exPlanation (SHAP), to develop a predictive model for critically ill patients. Methods We extracted data from the Medical Information Mart for Intensive Care IV database, encompassing all intensive care unit admissions. We employed nine different methods to develop the models. The most accurate model, with the highest area under the receiver operating characteristic curve, was selected as the optimal model. Additionally, we used SHAP to explain the workings of the ML model. Results The study included 21 395 critically ill patients, with a median age of 68 years (interquartile range, 56–79 years), and most patients were male (56.9%). The cohort was randomly split into a training set (N = 16 046) and a validation set (N = 5349). Among the nine models developed, the Random Forest model had the highest accuracy (87.62%) and the best area under the receiver operating characteristic curve value (0.89). The SHAP summary analysis showed that Glasgow Coma Scale, urine output, and blood urea nitrogen were the top three risk factors for outcome prediction. Furthermore, SHAP dependency analysis and SHAP force analysis were used to interpret the Random Forest model at the factor level and individual level, respectively. Conclusion A transparent ML model for predicting outcomes in critically ill patients using SHAP methodology is feasible and effective. SHAP values significantly improve the explainability of ML models.

medicine, general & internal
Abstract 13965: Natural Language Processing of Hospitalization Discharge Summary to Predict 1-year Post-Discharge Mortality Among Patients With Acute Heart Failure

Benjamin Rosand,Dragomir Radev,Harlan M Krumholz,Rohan Khera,Benjamin RosandDragomir RadevHarlan M KrumholzRohan KheraYale Sch of Medicine,New Haven,CTComputer Science,Yale Univ,New Haven,CTYale Univ,New Haven,CT

DOI: https://doi.org/10.1161/circ.144.suppl_1.13965

IF: 37.8

2021-11-10

Circulation

Abstract:Background: Patients hospitalized with heart failure (HF) have over 20% risk post-discharge 1-year mortality. Current risk models rely on multiple structured clinical data elements including vitals, diagnoses, laboratory tests and medications to predict mortality and have modest performance (AUROC 0.70-0.74). However, clinical notes, which capture the assessment of patients and their prognosis have had a limited role in predictive models, due to challenges with engineering features from the unstructured text. Methods: We identified all primary HF hospitalizations for consecutive patients at Yale, between 2013-2019, and linked to the Connecticut death index to assess 1-year post-discharge deaths. The discharge summaries for these hospitalizations were split into training and validation sets (80% and 10%), and a 10% independent test set. We used the interpretable term frequency-inverse document frequency NLP model to assess the probability of 1-year death, evaluated against observed events, presented as Kaplan-Meier Curves, and interpreted using a word importance plot. Results: There were 6702 discharge summaries for HF hospitalization survivors. Their mean age was 78 (SD 15) years, 50% were women, with a 1-year mortality of 21%. The NLP model on discharge summaries had high discrimination for 1-year death in the independent test set, AUROC 0.78 (Figure A), with high positive and negative predictive role (F1 score, 0.73). Moreover, the model identified prognosis over the course of the post-discharge year (Fig B), reflecting prognostic cues outside of the immediate post-discharge period. Word tokens of chronic, severe, end organ disease, represented most frequent predictors of mortality (Fig C). Conclusions: An NLP model exceeded the performance of the best models based on structured data. Risk-prediction using unstructured clinical notes could reduce the burden of data collection for risk prediction and adjustment while maintaining interpretability.

cardiac & cardiovascular systems,peripheral vascular disease
An Interpretable Deep-Learning Framework for Predicting Hospital Readmissions From Electronic Health Records

Fabio Azzalini,Tommaso Dolci,Marco Vagaggini

2023-10-16

Abstract:With the increasing availability of patients' data, modern medicine is shifting towards prospective healthcare. Electronic health records contain a variety of information useful for clinical patient description and can be exploited for the construction of predictive models, given that similar medical histories will likely lead to similar progressions. One example is unplanned hospital readmission prediction, an essential task for reducing hospital costs and improving patient health. Despite predictive models showing very good performances especially with deep-learning models, they are often criticized for the poor interpretability of their results, a fundamental characteristic in the medical field, where incorrect predictions might have serious consequences for the patient health. In this paper we propose a novel, interpretable deep-learning framework for predicting unplanned hospital readmissions, supported by NLP findings on word embeddings and by neural-network models (ConvLSTM) for better handling temporal data. We validate our system on the two predictive tasks of hospital readmission within 30 and 180 days, using real-world data. In addition, we introduce and test a model-dependent technique to make the representation of results easily interpretable by the medical staff. Our solution achieves better performances compared to traditional models based on machine learning, while providing at the same time more interpretable results.

Machine Learning,Information Retrieval
Interpretable machine learning for predicting 28-day all-cause in-hospital mortality for hypertensive ischemic or hemorrhagic stroke patients in the ICU: a multi-center retrospective cohort study with internal and external cross-validation

Jian Huang,Huaqiao Chen,Jiewen Deng,Xiaozhu Liu,Tingting Shu,Chengliang Yin,Minjie Duan,Li Fu,Kai Wang,Song Zeng

DOI: https://doi.org/10.3389/fneur.2023.1185447

IF: 3.4

2023-08-09

Frontiers in Neurology

Abstract:Background: Timely and accurate outcome prediction plays a critical role in guiding clinical decisions for hypertensive ischemic or hemorrhagic stroke patients admitted to the ICU. However, interpreting and translating the predictive models into clinical applications are as important as the prediction itself. This study aimed to develop an interpretable machine learning (IML) model that accurately predicts 28-day all-cause mortality in hypertensive ischemic or hemorrhagic stroke patients. Methods: A total of 4,274 hypertensive ischemic or hemorrhagic stroke patients admitted to the ICU in the USA from multicenter cohorts were included in this study to develop and validate the IML model. Five machine learning (ML) models were developed, including artificial neural network (ANN), gradient boosting machine (GBM), eXtreme Gradient Boosting (XGBoost), logistic regression (LR), and support vector machine (SVM), to predict mortality using the MIMIC-IV and eICU-CRD database in the USA. Feature selection was performed using the Least Absolute Shrinkage and Selection Operator (LASSO) algorithm. Model performance was evaluated based on the area under the curve (AUC), accuracy, positive predictive value (PPV), and negative predictive value (NPV). The ML model with the best predictive performance was selected for interpretability analysis. Finally, the SHapley Additive exPlanations (SHAP) method was employed to evaluate the risk of all-cause in-hospital mortality among hypertensive ischemic or hemorrhagic stroke patients admitted to the ICU. Results: The XGBoost model demonstrated the best predictive performance, with the AUC values of 0.822, 0.739, and 0.700 in the training, test, and external cohorts, respectively. The analysis of feature importance revealed that age, ethnicity, white blood cell (WBC), hyperlipidemia, mean corpuscular volume (MCV), glucose, pulse oximeter oxygen saturation (SpO 2 ), serum calcium, red blood cell distribution width (RDW), blood urea nitrogen (BUN), and bicarbonate were the 11 most important features. The SHAP plots were employed to interpret the XGBoost model. Conclusions: The XGBoost model accurately predicted 28-day all-cause in-hospital mortality among hypertensive ischemic or hemorrhagic stroke patients admitted to the ICU. The SHAP method can provide explicit explanations of personalized risk prediction, which can aid physicians in understanding the model.

neurosciences,clinical neurology
Distillation to Enhance the Portability of Risk Models Across Institutions with Large Patient Claims Database

Steve Nyemba,Chao Yan,Ziqi Zhang,Amol Rajmane,Pablo Meyer,Prithwish Chakraborty,Bradley Malin

DOI: https://doi.org/10.48550/arXiv.2207.02445

IF: 5.414

2022-07-06

Machine Learning

Abstract:Artificial intelligence, and particularly machine learning (ML), is increasingly developed and deployed to support healthcare in a variety of settings. However, clinical decision support (CDS) technologies based on ML need to be portable if they are to be adopted on a broad scale. In this respect, models developed at one institution should be reusable at another. Yet there are numerous examples of portability failure, particularly due to naive application of ML models. Portability failure can lead to suboptimal care and medical errors, which ultimately could prevent the adoption of ML-based CDS in practice. One specific healthcare challenge that could benefit from enhanced portability is the prediction of 30-day readmission risk. Research to date has shown that deep learning models can be effective at modeling such risk. In this work, we investigate the practicality of model portability through a cross-site evaluation of readmission prediction models. To do so, we apply a recurrent neural network, augmented with self-attention and blended with expert features, to build readmission prediction models for two independent large scale claims datasets. We further present a novel transfer learning technique that adapts the well-known method of born-again network (BAN) training. Our experiments show that direct application of ML models trained at one institution and tested at another institution perform worse than models trained and tested at the same institution. We further show that the transfer learning approach based on the BAN produces models that are better than those trained on just a single institution's data. Notably, this improvement is consistent across both sites and occurs after a single retraining, which illustrates the potential for a cheap and general model transfer mechanism of readmission risk prediction.
Predicting Mortality in Intensive Care Unit Patients With Heart Failure Using an Interpretable Machine Learning Model: Retrospective Cohort Study

Jili Li,Siru Liu,Yundi Hu,Lingfeng Zhu,Yujia Mao,Jialin Liu

DOI: https://doi.org/10.2196/38082

IF: 7.076

2022-08-10

Journal of Medical Internet Research

Abstract:Background: Heart failure (HF) is a common disease and a major public health problem. HF mortality prediction is critical for developing individualized prevention and treatment plans. However, due to their lack of interpretability, most HF mortality prediction models have not yet reached clinical practice. Objective: We aimed to develop an interpretable model to predict the mortality risk for patients with HF in intensive care units (ICUs) and used the SHapley Additive exPlanation (SHAP) method to explain the extreme gradient boosting (XGBoost) model and explore prognostic factors for HF. Methods: In this retrospective cohort study, we achieved model development and performance comparison on the eICU Collaborative Research Database (eICU-CRD). We extracted data during the first 24 hours of each ICU admission, and the data set was randomly divided, with 70% used for model training and 30% used for model validation. The prediction performance of the XGBoost model was compared with three other machine learning models by the area under the curve. We used the SHAP method to explain the XGBoost model. Results: A total of 2798 eligible patients with HF were included in the final cohort for this study. The observed in-hospital mortality of patients with HF was 9.97%. Comparatively, the XGBoost model had the highest predictive performance among four models with an area under the curve (AUC) of 0.824 (95% CI 0.7766-0.8708), whereas support vector machine had the poorest generalization ability (AUC=0.701, 95% CI 0.6433-0.7582). The decision curve showed that the net benefit of the XGBoost model surpassed those of other machine learning models at 10%~28% threshold probabilities. The SHAP method reveals the top 20 predictors of HF according to the importance ranking, and the average of the blood urea nitrogen was recognized as the most important predictor variable. Conclusions: The interpretable predictive model helps physicians more accurately predict the mortality risk in ICU patients with HF, and therefore, provides better treatment plans and optimal resource allocation for their patients. In addition, the interpretable framework can increase the transparency of the model and facilitate understanding the reliability of the predictive model for the physicians.

health care sciences & services,medical informatics
Towards Clinical Prediction with Transparency: An Explainable AI Approach to Survival Modelling in Residential Aged Care

Teo Susnjak,Elise Griffin

2023-12-08

Abstract:Background: Accurate survival time estimates aid end-of-life medical decision-making. Objectives: Develop an interpretable survival model for elderly residential aged care residents using advanced machine learning. Setting: A major Australasian residential aged care provider. Participants: Residents aged 65+ admitted for long-term care from July 2017 to August 2023. Sample size: 11,944 residents across 40 facilities. Predictors: Factors include age, gender, health status, co-morbidities, cognitive function, mood, nutrition, mobility, smoking, sleep, skin integrity, and continence. Outcome: Probability of survival post-admission, specifically calibrated for 6-month survival estimates. Statistical Analysis: Tested CoxPH, EN, RR, Lasso, GB, XGB, and RF models in 20 experiments with a 90/10 train/test split. Evaluated accuracy using C-index, Harrell's C-index, dynamic AUROC, IBS, and calibrated ROC. Chose XGB for its performance and calibrated it for 1, 3, 6, and 12-month predictions using Platt scaling. Employed SHAP values to analyze predictor impacts. Results: GB, XGB, and RF models showed the highest C-Index values (0.714, 0.712, 0.712). The optimal XGB model demonstrated a 6-month survival prediction AUROC of 0.746 (95% CI 0.744-0.749). Key mortality predictors include age, male gender, mobility, health status, pressure ulcer risk, and appetite. Conclusions: The study successfully applies machine learning to create a survival model for aged care, aligning with clinical insights on mortality risk factors and enhancing model interpretability and clinical utility through explainable AI.

Machine Learning
PNS196 USE OF MACHINE LEARNING IN HEALTH ECONOMICS AND OUTCOMES RESEARCH (HEOR): OPTIMIZING PREDICTIVE MODELS

J. Royer,E. Wu,R. Ayyagari

DOI: https://doi.org/10.1016/j.jval.2019.04.1553

IF: 5.156

2019-01-01

Value in Health

Abstract:Predictive modeling has a wide range of applications in HEOR. When the number of predictors is large in comparison to the number of observations, conventional generalized linear regression models can yield poor predictions. Machine Learning (ML) methods are increasingly being used in HEOR as an alternative framework to increase prediction accuracy. Commonly used ML models include Logistic-LASSO, tree based models (CART, Random Forest), and Neural Networks. This research aimed to explore the value of ML for predictive modeling in HEOR, using high dimensional datasets like genomic or claims, and the tradeoffs between model performance and interpretability. Although ML models often outperform regression-based approaches in prediction quality, questions remain about their interpretability. Comparisons between ML and conventional regression models were performed in three different settings: High Dimensional Propensity Score (HDPS) to estimate treatment effect, predicting treatment resistance using genomic data, and classifying patients for risk of an undiagnosed condition. Predictive performance of ML models including Logistic-LASSO, Random Forests, Neural Networks and a conventional logistic regression model were evaluated both in-sample and out-of-sample. Prediction accuracy (i.e. ROC-AUC), and bias (through simulations) were assessed for the different models. In each application, cross validation was used to minimize overfitting risk. ML models outperformed regression-based approaches in terms of prediction accuracy (increased ROC-AUC by ≥10%) and significantly decreased estimation bias (15% lower than regression in the HDPS application). However, there was a significant loss of interpretability, especially with Deep Neural Networks and Random Forests. Quantifying the impact of specific predictors on outcome is virtually impossible in these instances. ML models exhibited better predictive performance measured by ROC-AUC and bias than conventional logistic regression models at the expense of interpretability. Conventional regression models can provide a better framework if interpretability is central to the prediction analysis.

Effect of aflatoxin B1 on net synthesis of albumin, fibrinogen, and alpha-1-acid glycoprotein by the isolated perfused rat liver.

Interpretable (not just posthoc-explainable) heterogeneous survivor bias-corrected treatment effects for assignment of postdischarge interventions to prevent readmissions

Small samples-oriented intrinsically explainable machine learning using Variational Bayesian Logistic Regression: An intensive care unit readmission prediction case for liver transplantation patients

Hospital Readmission Prediction - Applying Hierarchical Sparsity Norms for Interpretable Models

The Price of Explainability in Machine Learning Models for 100-Day Readmission Prediction in Heart Failure: Retrospective, Comparative, Machine Learning Study

Interpretable machine learning models for hospital readmission prediction: a two-step extracted regression tree approach

Comparative analysis of explainable machine learning prediction models for hospital mortality

Predicting 30-day Hospital Readmission with Publicly Available Administrative Database. A Conditional Logistic Regression Modeling Approach

SCOPE: predicting future diagnoses in office visits using electronic health records

Leveraging deep survival models to predict quality of care risk in diverse hospital readmissions

Explainable machine learning for predicting conversion to neurological disease: Results from 52,939 medical records

Explainable Prediction of Adverse Outcomes Using Clinical Notes

Interpretable Neural Networks for Predicting Mortality Risk using Multi-modal Electronic Health Records

Explainable artificial intelligence model for mortality risk prediction in the intensive care unit: a derivation and validation study

Abstract 13965: Natural Language Processing of Hospitalization Discharge Summary to Predict 1-year Post-Discharge Mortality Among Patients With Acute Heart Failure

An Interpretable Deep-Learning Framework for Predicting Hospital Readmissions From Electronic Health Records

Interpretable machine learning for predicting 28-day all-cause in-hospital mortality for hypertensive ischemic or hemorrhagic stroke patients in the ICU: a multi-center retrospective cohort study with internal and external cross-validation

Distillation to Enhance the Portability of Risk Models Across Institutions with Large Patient Claims Database

Predicting Mortality in Intensive Care Unit Patients With Heart Failure Using an Interpretable Machine Learning Model: Retrospective Cohort Study

Towards Clinical Prediction with Transparency: An Explainable AI Approach to Survival Modelling in Residential Aged Care

PNS196 USE OF MACHINE LEARNING IN HEALTH ECONOMICS AND OUTCOMES RESEARCH (HEOR): OPTIMIZING PREDICTIVE MODELS