Abstract:Context: Many of the publicly available health quality report cards are based on administrative data. ICD-9-CM codes in administrative data are not date stamped to distinguish between medical conditions present at the time of hospital admission and complications, which occur after hospital admission. Treating complications as preexisting conditions gives poor-performing hospitals "credit" for their complications and may cause some hospitals that are delivering low-quality care to be misclassified as average- or high-performing hospitals. Objective: To determine whether hospital quality assessment based on administrative data is impacted by the inclusion of condition present at admission (CPAA) modifiers in administrative data as a date stamp indicator. Design, setting, and patients: Retrospective cohort study based on 648,866 inpatient admissions between 1998 and 2000 for coronary artery bypass graft (CABG) surgery, coronary angioplasty (PTCA), carotid endarterectomy (CEA), abdominal aortic aneurysm (AAA) repair, total hip replacement (THR), acute MI (AMI), and stroke using the California State Inpatient Database which includes CPAA modifiers. Hierarchical logistic regression was used to create separate condition-specific risk adjustment models. For each study population, one model was constructed using only secondary diagnoses present at admission based on the CPAA modifier: "date stamp" model. The second model was constructed using all secondary diagnoses, ignoring the information present in the CPAA modifier: the "no date stamp model." Hospital quality was assessed separately using the "date stamp" and the "no date stamp" risk-adjustment models. Results: Forty percent of the CABG hospitals, 33 percent of the PTCA hospitals, 40 percent of the THR hospitals, and 33 percent of the AMI hospitals identified as low-performance hospitals by the "date stamp" models were not classified as low-performance hospitals by the "no date stamp" models. Fifty percent of the CABG hospitals, 33 percent of the PTCA hospitals, 50 percent of the CEA hospitals, and 36 percent of the AMI hospitals identified as low-performance hospitals by the "no date stamp" models were not identified as low-performance hospitals by the "date stamp" models. The inclusion of the CPAA modifier had a minor impact on hospital quality assessment for AAA repair, stroke, and CEA. Conclusion: This study supports the hypothesis that the use of routine administrative data without date stamp information to construct hospital quality report cards may result in the mis-identification of hospital quality outliers. However, the CPAA modifier will need to be further validated before date stamped administrative data can be used as the basis for health quality report cards.

Administrative records-based criterion measures

The criterion problem: 1917–1992.

Adaptive performance: a criterion problem

Administrative Data for Quality Improvement

Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations

Meta-Analysis of Criterion Validity for Curriculum-Based Measurement in Written Language

Accuracy of hospital report cards based on administrative data

Dealing with the Criterion Problem by Measuring the Quality and Quantity of Creative Activity and Accomplishment

Basic statistical methods in determining criteria weights

Population-Level Administrative Data: A Resource to Advance Psychological Science

Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range.

Measurement invariance of Attention Deficit/Hyperactivity Disorder symptom criteria as rated by parents and teachers in children and adolescents: A systematic review

Measuring Classroom Management in Secondary Settings: Ongoing Validation of the Direct Behavior Rating-Classroom Management

A Cautionary Tail: A Framework and Case Study for Testing Predictive Model Validity

A scoping review of the methodological approaches used in retrospective chart reviews to validate adverse event rates in administrative data

How do we know we are measuring environmental attitude? Specific objectivity as the formal validation criterion for measures of latent attributes

Assessing the feasibility of anticholinergic burden scales and measures in administrative data: A systematic review

Primary care quality for older adults: Practice-based quality measures derived from a RAND/UCLA appropriateness method study

Cross-validating the Clinical Assessment of Attention Deficit–Adult symptom validity scales for assessment of attention deficit/hyperactivity disorder in adults

The Case Record Rating-Scale - A Method for Rating Symptom and Social Function Data from Case Records

Ecological validity of performance-based measures of executive functions: Is face validity necessary for prediction of daily functioning?