Assessing The Practice Of Data Quality Evaluation In A National Clinical Data Research Network Through A Systematic Scoping Review In The Era Of Real-World Data
Jiang Bian,Tianchen Lyu,Alexander Loiacono,Tonatiuh Mendoza Viramontes,Gloria Lipori,Yi Guo,Yonghui Wu,Mattia Prosperi,Thomas J George,Christopher A Harle,Elizabeth A Shenkman,William Hogan
DOI: https://doi.org/10.1093/jamia/ocaa245
2020-01-01
Journal of the American Medical Informatics Association
Abstract:Objective: To synthesize data quality (DQ) dimensions and assessment methods of real-world data, especially electronic health records, through a systematic scoping review and to assess the practice of DQ assessment in the national Patient-centered Clinical Research Network (PCORnet).Materials and Methods: We started with 3 widely cited DQ literature-2 reviews from Chan et al (2010) and Weiskopf et al (2013a) and 1 DQ framework from Kahn et al (2016)-and expanded our review systematically to cover relevant articles published up to February 2020. We extracted DQ dimensions and assessment methods from these studies, mapped their relationships, and organized a synthesized summarization of existing DQ dimensions and assessment methods. We reviewed the data checks employed by the PCORnet and mapped them to the synthesized DQ dimensions and methods.Results: We analyzed a total of 3 reviews, 20 DQ frameworks, and 226 DQ studies and extracted 14 DQ dimensions and 10 assessment methods. We found that completeness, concordance, and correctness/accuracy were commonly assessed. Element presence, validity check, and conformance were commonly used DQ assessment methods and were the main focuses of the PCORnet data checks.Discussion: Definitions of DQ dimensions and methods were not consistent in the literature, and the DQ assessment practice was not evenly distributed (eg, usability and ease-of-use were rarely discussed). Challenges in DQ assessments, given the complex and heterogeneous nature of real-world data, exist.Conclusion: The practice of DQ assessment is still limited in scope. Future work is warranted to generate understandable, executable, and reusable DQ measures.