Abstract:Background: Mental, emotional, and behavioral disorders are chronic pediatric conditions, and their prevalence has been on the rise over recent decades. Affected children have long-term health sequelae and a decline in health-related quality of life. Due to the lack of a validated database for pharmacoepidemiological research on selected mental, emotional, and behavioral disorders, there is uncertainty in their reported prevalence in the literature. objectives: We aimed to evaluate the accuracy of coding related to pediatric mental, emotional, and behavioral disorders in a large integrated health care system's electronic health records (EHRs) and compare the coding quality before and after the implementation of the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) coding as well as before and after the COVID-19 pandemic. Methods: Medical records of 1200 member children aged 2-17 years with at least 1 clinical visit before the COVID-19 pandemic (January 1, 2012, to December 31, 2014, the ICD-9-CM coding period; and January 1, 2017, to December 31, 2019, the ICD-10-CM coding period) and after the COVID-19 pandemic (January 1, 2021, to December 31, 2022) were selected with stratified random sampling from EHRs for chart review. Two trained research associates reviewed the EHRs for all potential cases of autism spectrum disorder (ASD), attention-deficit hyperactivity disorder (ADHD), major depression disorder (MDD), anxiety disorder (AD), and disruptive behavior disorders (DBD) in children during the study period. Children were considered cases only if there was a mention of any one of the conditions (yes for diagnosis) in the electronic chart during the corresponding time period. The validity of diagnosis codes was evaluated by directly comparing them with the gold standard of chart abstraction using sensitivity, specificity, positive predictive value, negative predictive value, the summary statistics of the F-score, and Youden J statistic. κ statistic for interrater reliability among the 2 abstractors was calculated. Results: The overall agreement between the identification of mental, behavioral, and emotional conditions using diagnosis codes compared to medical record abstraction was strong and similar across the ICD-9-CM and ICD-10-CM coding periods as well as during the prepandemic and pandemic time periods. The performance of AD coding, while strong, was relatively lower compared to the other conditions. The weighted sensitivity, specificity, positive predictive value, and negative predictive value for each of the 5 conditions were as follows: 100%, 100%, 99.2%, and 100%, respectively, for ASD; 100%, 99.9%, 99.2%, and 100%, respectively, for ADHD; 100%, 100%, 100%, and 100%, respectively for DBD; 87.7%, 100%, 100%, and 99.2%, respectively, for AD; and 100%, 100%, 99.2%, and 100%, respectively, for MDD. The F-score and Youden J statistic ranged between 87.7% and 100%. The overall agreement between abstractors was almost perfect (κ=95%). Conclusions: Diagnostic codes are quite reliable for identifying selected childhood mental, behavioral, and emotional conditions. The findings remained similar during the pandemic and after the implementation of the ICD-10-CM coding in the EHR system.

Identifying priorities in methodological research using ICD-9-CM and ICD-10 administrative data: report from an international consortium

Development of Data Quality Indicators for Improving Hospital International Classification of Diseases-Coded Health Data Quality Globally

Use of ICD-10-CM Codes for Adverse Social Determinants of Health Across Health Systems

A Comparative Research on the ICD-11 and ICD-10 Classification Systems

Best Practices for Identifying Hospitalized Lower Respiratory Tract Infections Using Administrative Data: A Systematic Literature Review of Validation Studies

A Comparative Research on the International Common Surgery and Operation Classification System

A New Hierarchical Approach to Requirement Analysis Used for ICD-10 Startup Index

Methodologies and key considerations for implementing the International Classification of Diseases-11th revision morbidity coding: insights from a national pilot study in China

Barriers to data quality resulting from the process of coding health information to administrative data: a qualitative study

Health data quality improvement by comparing administrative medical data and billing data

A methodology for identifying high-need, high-cost patient personas for international comparisons

A retrospective analysis using comorbidity detecting algorithmic software to determine the incidence of International Classification of Diseases (ICD) code omissions and appropriateness of Diagnosis-Related Group (DRG) code modifiers

State and Government Administrative Databases: Medicare, National Inpatient Sample (NIS), and State Inpatient Databases (SID) Programs

Further Understanding of the Importance of the Coding Work of the International Classification of Diseases

Field testing a new ICD coding system: methods and early experiences with ICD-11 Beta Version 2018

Coding of Childhood Psychiatric and Neurodevelopmental Disorders in Electronic Health Records of a Large Integrated Health Care System: Validation Study

A one stop shop for cost-effectiveness evidence? Recommendations for improving Disease Control Priorities

Adoption of network and plan-do-check-action in the international classification of disease 10 coding

Identifying Conditions With High Prevalence, Cost, and Variation in Cost in US Children’s Hospitals

Coding rules for uncertain and "ruled out" diagnoses in ICD-10 and ICD-11

A Survey of Automated ICD Coding: Development, Challenges, and Applications