Abstract:Background: Implementation support practitioners (ISPs) are professionals that support others to implement evidence-informed practices, programs, and policies in various service delivery settings to achieve population outcomes. Measuring the use of competencies by ISPs provides a unique opportunity to assess an understudied facet of implementation science-how knowledge, attitudes, and skills used by ISPs affects sustainable change in complicated and complex service systems. This study describes the development and validation of a measure-the Implementation Support Competencies Assessment (ISCA)-that assesses implementation support competencies, with versatile applications across service contexts. Methods: Recently developed practice guide materials included operationalizations of core competencies for ISPs across three domains: co-creation and engagement, ongoing improvement, and sustaining change. These operationalizations, in combination with recent empirical and conceptual work, provided an initial item pool and foundation on which to advance measurement development, largely from a confirmatory perspective (as opposed to exploratory). The measure was further refined through modified cognitive interviewing with three highly experienced ISPs and pilot-testing with 39 individuals enrolled in a university-based certificate program in implementation practice. To recruit a sample for validation analyses, we leveraged a listserv of nearly 4,000 individuals who have registered for or expressed interest in various events and trainings focused on implementation practice offered by an implementation science collaborative housed within a research-intensive university in the Southeast region of the United States. Our final analytic sample included 357 participants who self-identified as ISPs. Results: Assessments of internal consistency reliability for each competency-specific item set yielded evidence of strong reliability. Results from confirmatory factor analyses provided evidence for the factorial and construct validity of all three domains and associated competencies in the ISCA. Conclusions: The findings suggest that one's possession of high levels of competence across each of the three competency domains is strongly associated with theorized outcomes that can promote successful and sustainable implementation efforts among those who receive implementation support from an ISP. The ISCA serves as a foundational tool for workforce development to formally measure and assess improvement in the skills that are required to tailor a package of implementation strategies situated in context.

The Assessment of Post-secondary Supports: A New Measure of an Understudied SCCT Construct

Development and Validation of a Short Form of the College-Going Self-Efficacy Scale

Parental Support, Critical Consciousness, and Agency in Career Decision Making for Urban Students

Precision in Career Motivation Assessment

Exploring Students' Perceived Constraints Support in Physical Education: Measurement Development and Learning Outcomes

An Examination of the Applicability of Social Cognitive Career Theory for African American College Students

The Self-efficacy for Parenting Adolescents Scale: Development and Initial Validation

Development of a Brief Coparenting Measure: The Coparenting Competence Scale

The Career Support Scale for Women in Engineering (CSS-WE): Development and Validation With Engineering Students in Korea

Self-Efficacy Regarding Social Work Competencies

Development and psychometric evaluation of the Implementation Support Competencies Assessment

Understanding and Assessing Construct Validity of the Social Provisions Scale: Implications for Youth Development

Measuring Cultural Competencies: The Development and Validation of a Revised Measure of Sociocultural Adaptation

Development and Validation of the Sense of Online Community Scale

Validation of a youth social capital scale Informed by positive youth development

Enhancement and Standardization of a Universal Social-Emotional Health Measure for Students’ Psychological Strengths

The Counseling Center Assessment of Psychological Symptoms (CCAPS): Merging clinical practice, training, and research.

A Bottom-Up Approach to Measuring Student Social–Emotional Competence

Examining an Integrated Factor Structure of Schoolwide MTSS Implementation Measures

Development of the Short Critical Consciousness Scale (ShoCCS)

Development and Validation of a Teacher Report Measure for Assessing Social-Emotional Strengths of Children and Adolescents