Abstract:This manuscript presents a proof-of-concept for a generalizable strategy, the full algorithm, designed to estimate disease risk using real-world clinical tabular data systems, such as electronic health records (EHR) or claims databases. By integrating classic statistical methods and modern artificial intelligence techniques, this strategy automates the production of a disease prediction model that comprehensively reflects the dynamics contained within the underlying data system. Specifically, the full algorithm parses through every facet of the data (e.g., encounters, diagnoses, procedures, medications, labs, chief complaints, flowsheets, vital signs, demographics, etc.), selects which factors to retain as predictor variables by evaluating the data empirically against statistical criteria, structures and formats the retained data into time-series, trains a neural network-based prediction model, then subsequently applies this model to current patients to generate risk estimates. A distinguishing feature of the proposed strategy is that it produces a self-adaptive prediction system, capable of evolving the prediction mechanism in response to changes within the data: as newly collected data expand/modify the dataset organically, the prediction mechanism automatically evolves to reflect these changes. Moreover, the full algorithm operates without the need for a-priori data curation and aims to harness all informative risk and protective factors within the real-world data. This stands in contrast to traditional approaches, which often rely on highly curated datasets and domain expertise to build static prediction models based solely on well-known risk factors. As a proof-of-concept, we codified the full algorithm and tasked it with estimating 12-month risk of initial stroke or myocardial infarction using our hospital's real-world EHR. A 66-month pseudo-prospective validation was conducted using records from 558,105 patients spanning April 2015 to September 2023, totalling 3,424,060 patient-months. Area under the receiver operating characteristic curve (AUROC) values ranged from .830 to .909, with an improving trend over time. Odds ratios describing model precision for patients 1-100 and 101-200 (when ranked by estimated risk) ranged from 15.3 to 48.1 and 7.2 to 45.0, respectively, with both groups showing improving trends over time. Findings suggest the feasibility of developing high-performing disease risk calculators in the proposed manner.

SCOPE: predicting future diagnoses in office visits using electronic health records

A Patient-Similarity-based Model for Diagnostic Prediction

A Method for the Early Prediction of Chronic Diseases Based on Short Sequential Medical Data.

Predicting Chronic Disease Hospitalizations from Electronic Health Records: An Interpretable Classification Approach

Predicting disease onset from electronic health records for population health management: a scalable and explainable Deep Learning approach

Prediction of future hospital admissions - what is the tradeoff between specificity and accuracy?

Scalable and accurate deep learning with electronic health records

Incorporating Medical Code Descriptions for Diagnosis Prediction in Healthcare

In-hospital mortality, readmission, and prolonged length of stay risk prediction leveraging historical electronic patient records

Label-dependent and Event-Guided Interpretable Disease Risk Prediction Using EHRs

Exploring Long-Term Prediction of Type 2 Diabetes Microvascular Complications

Towards Reducing Diagnostic Errors with Interpretable Risk Prediction

A General Framework for Diagnosis Prediction Via Incorporating Medical Code Descriptions

Towards artificial intelligence-based disease prediction algorithms that comprehensively leverage and continuously learn from real-world clinical tabular data systems

Sequential Inference of Hospitalization Electronic Health Records Using Probabilistic Models

Explainable machine learning for predicting conversion to neurological disease: Results from 52,939 medical records

A Knowledge Distillation Ensemble Framework for Predicting Short and Long-term Hospitalisation Outcomes from Electronic Health Records Data

Electronic Health Records-Based Data-Driven Diabetes Knowledge Unveiling and Risk Prognosis

Language-model-based patient embedding using electronic health records facilitates phenotyping, disease forecasting, and progression analysis

Automated feature selection of predictors in electronic medical records data

Continuous Predictive Modeling of Clinical Notes and ICD Codes in Patient Health Records