Abstract:This manuscript presents a proof-of-concept for a generalizable strategy, the full algorithm, designed to estimate disease risk using real-world clinical tabular data systems, such as electronic health records (EHR) or claims databases. By integrating classic statistical methods and modern artificial intelligence techniques, this strategy automates the production of a disease prediction model that comprehensively reflects the dynamics contained within the underlying data system. Specifically, the full algorithm parses through every facet of the data (e.g., encounters, diagnoses, procedures, medications, labs, chief complaints, flowsheets, vital signs, demographics, etc.), selects which factors to retain as predictor variables by evaluating the data empirically against statistical criteria, structures and formats the retained data into time-series, trains a neural network-based prediction model, then subsequently applies this model to current patients to generate risk estimates. A distinguishing feature of the proposed strategy is that it produces a self-adaptive prediction system, capable of evolving the prediction mechanism in response to changes within the data: as newly collected data expand/modify the dataset organically, the prediction mechanism automatically evolves to reflect these changes. Moreover, the full algorithm operates without the need for a-priori data curation and aims to harness all informative risk and protective factors within the real-world data. This stands in contrast to traditional approaches, which often rely on highly curated datasets and domain expertise to build static prediction models based solely on well-known risk factors. As a proof-of-concept, we codified the full algorithm and tasked it with estimating 12-month risk of initial stroke or myocardial infarction using our hospital's real-world EHR. A 66-month pseudo-prospective validation was conducted using records from 558,105 patients spanning April 2015 to September 2023, totalling 3,424,060 patient-months. Area under the receiver operating characteristic curve (AUROC) values ranged from .830 to .909, with an improving trend over time. Odds ratios describing model precision for patients 1-100 and 101-200 (when ranked by estimated risk) ranged from 15.3 to 48.1 and 7.2 to 45.0, respectively, with both groups showing improving trends over time. Findings suggest the feasibility of developing high-performing disease risk calculators in the proposed manner.

Risk Projection for Time-to-Event Outcome Leveraging Summary Statistics With Source Individual-Level Data

Risk projection for time-to-event outcome from population-based case–control studies leveraging summary statistics from the target population

Efficient Risk Assessment of Time‐to‐Event Targets With Adaptive Information Transfer

Risk Prediction with Imperfect Survival Outcome Information from Electronic Health Records

Risk prediction for prostate cancer recurrence through regularized estimation with simultaneous adjustment for nonlinear clinical effects

Learning from Local to Global - an Efficient Distributed Algorithm for Modeling Time-to-event Data

Leveraging External Aggregated Information for the Marginal Accelerated Failure Time Model

Combining isotonic regression and EM algorithm to predict genetic risk under monotonicity constraint

Semiparametric model averaging prediction for lifetime data via hazards regression

Recurrent Events Analysis With Data Collected at Informative Clinical Visits in Electronic Health Records

Time-varying effect in the competing risks based on restricted mean time lost

Time-varying effect in older patients with early-stage breast cancer: a model considering the competing risks based on a time scale

Renewable risk assessment of heterogeneous streaming time‐to‐event cohorts

Dynamic Prediction of Competing Risk Events using Landmark Sub-distribution Hazard Model with Multiple Longitudinal Biomarker

Accommodating time-varying heterogeneity in risk estimation under the Cox model: a transfer learning approach

Towards artificial intelligence-based disease prediction algorithms that comprehensively leverage and continuously learn from real-world clinical tabular data systems

Heterogeneous individual risk modelling of recurrent events

Data-integration with pseudoweights and survey-calibration: application to developing US-representative lung cancer risk models for use in screening

A Threshold-free Prospective Prediction Accuracy Measure for Censored Time to Event Data

Personalized Risk Prediction for Cancer Survivors: A Bayesian Semi-parametric Recurrent Event Model with Competing Outcomes

Long-term effect estimation when combining clinical trial and observational follow-up datasets