Abstract:Background: The promise of real-world evidence and the learning health care system primarily depends on access to high-quality data. Despite widespread awareness of the prevalence and potential impacts of poor data quality (DQ), best practices for its assessment and improvement are unknown. Objective: This review aims to investigate how existing research studies define, assess, and improve the quality of structured real-world health care data. Methods: A systematic literature search of studies in the English language was implemented in the Embase and PubMed databases to select studies that specifically aimed to measure and improve the quality of structured real-world data within any clinical setting. The time frame for the analysis was from January 1945 to June 2023. We standardized DQ concepts according to the Data Management Association (DAMA) DQ framework to enable comparison between studies. After screening and filtering by 2 independent authors, we identified 39 relevant articles reporting DQ improvement initiatives. Results: The studies were characterized by considerable heterogeneity in settings and approaches to DQ assessment and improvement. Affiliated institutions were from 18 different countries and 18 different health domains. DQ assessment methods were largely manual and targeted completeness and 1 other DQ dimension. Use of DQ frameworks was limited to the Weiskopf and Weng (3/6, 50%) or Kahn harmonized model (3/6, 50%). Use of standardized methodologies to design and implement quality improvement was lacking, but mainly included plan-do-study-act (PDSA) or define-measure-analyze-improve-control (DMAIC) cycles. Most studies reported DQ improvements using multiple interventions, which included either DQ reporting and personalized feedback (24/39, 61%), IT-related solutions (21/39, 54%), training (17/39, 44%), improvements in workflows (5/39, 13%), or data cleaning (3/39, 8%). Most studies reported improvements in DQ through a combination of these interventions. Statistical methods were used to determine significance of treatment effect (22/39, 56% times), but only 1 study implemented a randomized controlled study design. Variability in study designs, approaches to delivering interventions, and reporting DQ changes hindered a robust meta-analysis of treatment effects. Conclusions: There is an urgent need for standardized guidelines in DQ improvement research to enable comparison and effective synthesis of lessons learned. Frameworks such as PDSA learning cycles and the DAMA DQ framework can facilitate this unmet need. In addition, DQ improvement studies can also benefit from prioritizing root cause analysis of DQ issues to ensure the most appropriate intervention is implemented, thereby ensuring long-term, sustainable improvement. Despite the rise in DQ improvement studies in the last decade, significant heterogeneity in methodologies and reporting remains a challenge. Adopting standardized frameworks for DQ assessment, analysis, and improvement can enhance the effectiveness, comparability, and generalizability of DQ improvement initiatives.

Development of a Methodological Approach for Data Quality Ontology in Diabetes Management

Ontology for Data Quality and Chronic Disease Management

Towards an ontology for data quality in integrated chronic disease management: A realist review of the literature.

An Ontology-Based Approach to Patient Follow-Up Assessment for Continuous and Personalized Chronic Disease Management

Integrating HL7 RIM and Ontology for Unified Knowledge and Data Representation in Clinical Decision Support Systems

Ontological specification of quality of chronic disease data in EHRs to support decision analytics: a realist review

Validating an Ontology-Based Algorithm to Identify Patients with Type 2 Diabetes Mellitus in Electronic Health Records

Assessing The Practice Of Data Quality Evaluation In A National Clinical Data Research Network Through A Systematic Scoping Review In The Era Of Real-World Data

Developing a Data Element Repository to Support EHR-driven Phenotype Algorithm Authoring and Execution

A Harmonized Data Quality Assessment Terminology and Framework for the Secondary Use of Electronic Health Record Data

Data Quality-Driven Improvement in Health Care: Systematic Literature Review

Quality assessment of real-world data repositories across the data life cycle: A literature review

Development and application of Chinese medical ontology for diabetes mellitus

Telehealth System Based on the Ontology Design of a Diabetes Management Pathway Model in China: Development and Usability Study.

Ontology-Oriented Diagnostic System for Traditional Chinese Medicine Based on Relation Refinement

Automating Electronic Health Record Data Quality Assessment

Systematically assessing the quality of dental electronic health record data for an investigation into oral health care disparities

Assessing the Effect of Electronic Health Record Data Quality on Identifying Patients With Type 2 Diabetes: Cross-Sectional Study

Development and Validation of ML-DQA -- a Machine Learning Data Quality Assurance Framework for Healthcare

Extending an open-source tool to measure data quality: case report on Observational Health Data Science and Informatics (OHDSI)

A telehealth system based on the ontology design of a diabetes management pathway model in China: Development and evaluation (Preprint)