Abstract:BACKGROUND In emergency departments (ED), timely rescue is very important as patients’ conditions usually deteriorate rapidly. Early diagnosis can increase patients’ chances of survival. Early diagnosis can be improved by predictive models based on machine learning using Electronic Medical Record (EMR) data. However, ED data are usually imbalanced, having missing values and sparse features. These quality issues make it challenging to build early identification models for diseases in ED. OBJECTIVE The objective of this study is to propose a systematic approach to deal with missing, imbalanced and sparse feature problems of ED data. METHODS We used random forest and K-means algorithms to interpolate missing values and under-sample data. Regarding sparse features, we used principal component analysis to reduce dimensions. For continuous and discrete variables, the decision coefficient R2 and Kappa coefficient are used to evaluate the performance respectively. The area under the receiver operating characteristic curve (AUC) and the area under the precision-recall curve (AUPRC) are used to estimate the model performance. To further evaluate the proposed approach, we carried out a case study using an ED dataset extracted from Hainan Hospital of Chinese PLA General Hospital. A logistic regression model for patient condition worsening prediction was built out of the data processed by the proposed approach. RESULTS A total of 1085 patients with rescue record and 17959 patients without rescue record were collected, which were significantly imbalanced. 275, 402 and 891 variables are extracted from laboratory tests, medications and diagnosis, respectively. After data preprocessing, the median R2 of random forest interpolation for continuous variables is 0.623 (IQR: 0.647), and the median of Kappa coefficient for discrete variable interpolation is 0.444 (IQR: 0.285). The logistic regression model constructed using the initial diagnostic data has poor performance and variable separation, which is reflected in the abnormally high OR values of the two variables of cardiac arrest and respiratory arrest (27857.4 and 9341.6) and an abnormal confidence interval. Using the processed data, the recall of the model reaches 0.77, F1-SCORE is 0.74, and AUC is 0.64. CONCLUSIONS We proposed a machine learning method to deal with data quality issues such as missing data, data imbalance, and sparse features in emergency data, so as to improve data availability. A preliminary case study indicate the results produced by the proposed method can be used for building prediction model for emergency patients.

Post-surgical Complication Prediction in the Presence of Low-Rank Missing Data.

Postoperative Survival Prediction Model of BP Neural Network with Variable Cluster

Imputation and low-rank estimation with Missing Not At Random data

Predictive analytics with ensemble modeling in laparoscopic surgery: A technical note

Explainable Machine-Learning Predictions for Complications after Pediatric Congenital Heart Surgery

Handling the Missing Data Problem in Electronic Health Records for Cancer Prediction.

Mining for equitable health: Assessing the impact of missing data in electronic health records

Temporal Matrix Completion with Locally Linear Latent Factors for Medical Applications

Matrix Completion under Low-Rank Missing Mechanism

Matrix Completion for Survey Data Prediction with Multivariate Missingness

Federated learning model for predicting major postoperative complications

Strategies for handling missing clinical data for automated surgical site infection detection from the electronic health record

Successively Alternate Least Square for Low-Rank Matrix Factorization with Bounded Missing Data

Dynamic Predictions of Postoperative Complications from Explainable, Uncertainty-Aware, and Multi-Task Deep Neural Networks

Added Value of Intraoperative Data for Predicting Postoperative Complications: Development and Validation of a MySurgeryRisk Extension

Multilevel Stochastic Optimization for Imputation in Massive Medical Data Records

Prognostics of Surgical Site Infections using Dynamic Health Data

An accurate missing data prediction method using LSTM based deep learning for health care

Dealing with the Missing, Imbalanced and Sparse Features Problems in Emergency Data Using Random Forest, K-means and PCA Respectively (Preprint)

Dealing with Missing, Imbalanced, and Sparse Features During the Development of a Prediction Model for Sudden Death Using Emergency Medicine Data: Machine Learning Approach

Integrated Convolutional and Recurrent Neural Networks for Health Risk Prediction using Patient Journey Data with Many Missing Values