Abstract:The recent release of large-scale healthcare datasets has greatly propelled the research of data-driven deep learning models for healthcare applications. However, due to the nature of such deep black-boxed models, concerns about interpretability, fairness, and biases in healthcare scenarios where human lives are at stake call for a careful and thorough examination of both datasets and models. In this work, we focus on MIMIC-IV (Medical Information Mart for Intensive Care, version IV), the largest publicly available healthcare dataset, and conduct comprehensive analyses of interpretability as well as dataset representation bias and prediction fairness of deep learning models for in-hospital mortality prediction. First, we analyze the interpretability of deep learning mortality prediction models and observe that (1) the best-performing interpretability method successfully identifies critical features for mortality prediction on various prediction models as well as recognizing new important features that domain knowledge does not consider; (2) prediction models rely on demographic features, raising concerns in fairness. Therefore, we then evaluate the fairness of models and do observe the unfairness: (1) there exists disparate treatment in prescribing mechanical ventilation among patient groups across ethnicity, gender and age; (2) models often rely on racial attributes unequally across subgroups to generate their predictions. We further draw concrete connections between interpretability methods and fairness metrics by showing how feature importance from interpretability methods can be beneficial in quantifying potential disparities in mortality predictors. Our analysis demonstrates that the prediction performance is not the only factor to consider when evaluating models for healthcare applications, since high prediction performance might be the result of unfair utilization of demographic features. Our findings suggest that future research in AI models for healthcare applications can benefit from utilizing the analysis workflow of interpretability and fairness as well as verifying if models achieve superior performance at the cost of introducing bias.

Addressing Noise And Skewness In Interpretable Health-Condition Assessment By Learning Model Confidence

DeepHealth: Deep Representation Learning with Autoencoders for Healthcare Prediction

Interpretable Clinical Prediction Via Attention-Based Neural Network.

Assisting Schizophrenia Diagnosis Using Clinical Electroencephalography and Interpretable Graph Neural Networks: a Real-World and Cross-Site Study

A Deep Learning Model Incorporating Knowledge Representation Vectors and Its Application in Diabetes Prediction

Care for the Mind Amid Chronic Diseases: An Interpretable AI Approach Using IoT

AdaCare: Explainable Clinical Health Status Representation Learning via Scale-Adaptive Feature Extraction and Recalibration

Health-LLM: Personalized Retrieval-Augmented Disease Prediction Model

Few-Shot Learning for Chronic Disease Management: Leveraging Large Language Models and Multi-Prompt Engineering with Medical Knowledge Injection

Interpretability and fairness evaluation of deep learning models on MIMIC-IV dataset

An interpretable machine learning framework for diagnosis and prognosis of COVID-19

Enhancing trustworthiness and reliability: advance explainable artificial intelligence framework for real world Sclerosis detection

Interpretable machine learning for dermatological disease detection: Bridging the gap between accuracy and explainability

Explainable Diagnosis Prediction through Neuro-Symbolic Integration

Health-LLM: Personalized Retrieval-Augmented Disease Prediction System

The detection of mental health conditions by incorporating external knowledge

Distilling Knowledge from Deep Networks with Applications to Healthcare Domain

Intelligent medical diagnosis and treatment for diabetes with deep convolutional fuzzy neural networks

Deep Learning Applied to Chest X-Rays: Exploiting and Preventing Shortcuts

A survey on the interpretability of deep learning in medical diagnosis

Removing Confounding Factors Associated Weights in Deep Neural Networks Improves the Prediction Accuracy for Healthcare Applications