Abstract:When dealing with a newly emerging disease such as COVID-19, the impact of patient- and disease-specific factors (e.g., body weight or known co-morbidities) on the immediate course of the disease is largely unknown. An accurate prediction of the most likely individual disease progression can improve the planning of limited resources and finding the optimal treatment for patients. In the case of COVID-19, the need for intensive care unit (ICU) admission of pneumonia patients can often only be determined on short notice by acute indicators such as vital signs (e.g., breathing rate, blood oxygen levels), whereas statistical analysis and decision support systems that integrate all of the available data could enable an earlier prognosis. To this end, we propose a holistic, multimodal graph-based approach combining imaging and non-imaging information. Specifically, we introduce a multimodal similarity metric to build a population graph that shows a clustering of patients. For each patient in the graph, we extract radiomic features from a segmentation network that also serves as a latent image feature encoder. Together with clinical patient data like vital signs, demographics, and lab results, these modalities are combined into a multimodal representation of each patient. This feature extraction is trained end-to-end with an image-based Graph Attention Network to process the population graph and predict the COVID-19 patient outcomes: admission to ICU, need for ventilation, and mortality. To combine multiple modalities, radiomic features are extracted from chest CTs using a segmentation neural network. Results on a dataset collected in Klinikum rechts der Isar in Munich, Germany and the publicly available iCTCF dataset show that our approach outperforms single modality and non-graph baselines. Moreover, our clustering and graph attention increases understanding of the patient relationships within the population graph and provides insight into the network's decision-making process.

Time-aware Context-Gated Graph Attention Network for Clinical Risk Prediction

Extracting Dynamic Information of Temporal Clinical Data to Predict the Outcome in Critically Ill Patients.

Generating Synthetic Mixed-Type Longitudinal Electronic Health Records for Artificial Intelligent Applications

Interpretable time-aware and co-occurrence-aware network for medical prediction

Time-aware Heterogeneous Graph Transformer with Adaptive Attention Merging for Health Event Prediction

Graph neural networks for clinical risk prediction based on electronic health records: A survey

Temporal-spatial Correlation Attention Network for Clinical Data Analysis in Intensive Care Unit

Modeling Long-term Dependencies and Short-term Correlations in Patient Journey Data with Temporal Attention Networks for Health Prediction

Temporal-Aware Graph Neural Network for Credit Risk Prediction.

StageNet: Stage-Aware Neural Networks for Health Risk Prediction

U-GAT: Multimodal Graph Attention Network for COVID-19 Outcome Prediction

Predictive Modeling with Temporal Graphical Representation on Electronic Health Records

A novel interactive deep cascade spectral graph convolutional network with multi-relational graphs for disease prediction

Graph learning with label attention and hyperbolic embedding for temporal event prediction in healthcare

A hybrid method of recurrent neural network and graph neural network for next-period prescription prediction

TAGNet - Temporal Aware Graph Convolution Network for Clinical Information Extraction.

Predicting Physiological Response in Heart Failure Management: A Graph Representation Learning Approach using Electronic Health Records

Multimodal graph attention network for COVID-19 outcome prediction

Learning Hierarchical Representations of Electronic Health Records for Clinical Outcome Prediction.

Context-Aware Health Event Prediction via Transition Functions on Dynamic Disease Graphs

Self-explainable Graph Neural Network for Alzheimer's Disease And Related Dementias Risk Prediction