Abstract:BackgroundImplementation outcome measures are essential for monitoring and evaluating the success of implementation efforts. Yet, currently available measures lack conceptual clarity and have largely unknown reliability and validity. This study developed and psychometrically assessed three new measures: the Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM).MethodsThirty-six implementation scientists and 27 mental health professionals assigned 31 items to the constructs and rated their confidence in their assignments. The Wilcoxon one-sample signed rank test was used to assess substantive and discriminant content validity. Exploratory and confirmatory factor analysis (EFA and CFA) and Cronbach alphas were used to assess the validity of the conceptual model. Three hundred twenty-six mental health counselors read one of six randomly assigned vignettes depicting a therapist contemplating adopting an evidence-based practice (EBP). Participants used 15 items to rate the therapist’s perceptions of the acceptability, appropriateness, and feasibility of adopting the EBP. CFA and Cronbach alphas were used to refine the scales, assess structural validity, and assess reliability. Analysis of variance (ANOVA) was used to assess known-groups validity. Finally, half of the counselors were randomly assigned to receive the same vignette and the other half the opposite vignette; and all were asked to re-rate acceptability, appropriateness, and feasibility. Pearson correlation coefficients were used to assess test-retest reliability and linear regression to assess sensitivity to change.ResultsAll but five items exhibited substantive and discriminant content validity. A trimmed CFA with five items per construct exhibited acceptable model fit (CFI = 0.98, RMSEA = 0.08) and high factor loadings (0.79 to 0.94). The alphas for 5-item scales were between 0.87 and 0.89. Scale refinement based on measure-specific CFAs and Cronbach alphas using vignette data produced 4-item scales (α’s from 0.85 to 0.91). A three-factor CFA exhibited acceptable fit (CFI = 0.96, RMSEA = 0.08) and high factor loadings (0.75 to 0.89), indicating structural validity. ANOVA showed significant main effects, indicating known-groups validity. Test-retest reliability coefficients ranged from 0.73 to 0.88. Regression analysis indicated each measure was sensitive to change in both directions.ConclusionsThe AIM, IAM, and FIM demonstrate promising psychometric properties. Predictive validity assessment is planned.

The Patient-Perceived Helpfulness of Measures Scale: Development and Validation of a Scale to Assess the Helpfulness of Using Measures in Psychological Treatment

The Partners in Health scale: The development and psychometric properties of a generic assessment scale for chronic condition self-management

Psychometric properties of the Positive Mental Health Scale (PMH-scale)

The internal consistency and construct validity of the partners in health scale: validation of a patient rated chronic condition self-management measure

Validation of the shortened Perceived Medical Condition Self-Management Scale in patients with chronic disease.

The Therapy Progress Scale: Evaluating Psychometric Properties in an Outpatient Sample of Clients in Private Practice

A new complex mental health test in a positive psychological framework

Checklist to operationalize measurement characteristics of patient-reported outcome measures

Item development for a patient‐reported measure of compassionate healthcare in action

Development and Psychometric Validation of the Hospitalized Patients’ Expectations for Treatment Scale-Patient Version

Development and Validation of the Chronic Condition Physician-Patient Relationship Scale (CC-PPR): A patient-informed measurement tool.

A review and recommendations for optimal outcome measures of anxiety, depression and general distress in studies evaluating psychosocial interventions for English-speaking adults with heterogeneous cancer diagnoses

Assessing Children's Eudaimonic Well-Being: The PROMIS Pediatric Meaning and Purpose Item Banks

Development and psychometric validation of the hospitalized patients' expectations for treatment scale-clinician version

Psychometric assessment of three newly developed implementation outcome measures

Measurement properties of patient-reported outcome measures of mental help-seeking attitude: a systematic review of psychometric properties

Psychometric Testing of a Patient Empowerment, Engagement, and Activation Survey

A patient-reported experience measure for community pharmacy including development of a short-form: The perceived service quality scale

Measuring social health in the patient-reported outcomes measurement information system (PROMIS): item bank development and testing

Assessment of the psychometrics of a PROMIS item bank: self-efficacy for managing daily activities

The Mental Health Literacy Scale (MHLS): A new scale-based measure of mental health literacy