Abstract:Though treatment integrity measurement is important for research intended to promote social and behavioral outcomes of children at risk for emotional and behavioral disorders (EBDs) in early childhood settings, measurement gaps exist in the field. This paper reports on the development and preliminary psychometric assessment of the treatment integrity measure for early childhood settings (TIMECS), an observational measure designed to address existing measurement gaps related to treatment integrity with tier 2 interventions in the early childhood field. To assess the preliminary score reliability (interrater) and validity (construct, discriminant) of the TIMECS, live observations (N = 650) in early childhood classrooms from 54 teachers (92.6% female, 7.4% male; 61.1% White) and 91 children (M age = 4.53 years, SD = .44; 45.1% female, 54.9% male; 45.1% Black) at risk for EBDs were scored by 12 coders using the TIMECS and an observational measure designed to assess teacher-child interactions. Teachers also self-reported on the quality of the teacher-child relationship. Interrater reliability (intraclass correlation coefficients, ICC [2,2]) for the quantity (i.e., adherence) item scores had a mean of .81 (SD = .07; range from .68 to .95), and the quality (i.e., competence) item scores had a mean of .69 (SD = .08; range from .52 to .80). Scores on the TIMECS Quantity and Quality items and scales showed evidence of construct validity, with the magnitude of the correlations suggesting that the quantity and quality items assess distinct components of treatment integrity. A TIMECS quantity scale also showed promise for intervention evaluation research by discriminating between teachers who had and had not been trained in a specific evidence-based intervention targeting social and behavioral skills in early childhood. The findings support the potential of the TIMECS to assess treatment integrity of teacher-delivered practices designed to address child social and behavioral outcomes of children at risk for EBDs in early childhood settings.

Treatment Integrity of Interventions With Children in the School Psychology Literature from 1995 to 2008

A Treatment Integrity Analysis of Function-Based Intervention

Treatment Integrity of Primary (Tier 1) Prevention Efforts in Tiered Systems: Mapping the Literature

Examining fidelity reporting within studies of foundational reading interventions for elementary students with or at risk for dyslexia

Development and Initial Psychometrics of a Generic Treatment Integrity Measure Designed to Assess Practice Elements Targeting Social, Emotional, and Behavioral Outcomes in Early Childhood Settings

Fidelity in Sensory Integration Intervention Research

The role of treatment fidelity on outcomes during a randomized field trial of an autism intervention

Developing Treatments for Antisocial Behavior Among Children: Controlled Trials and Uncontrolled Tribulations

The relationship between treatment integrity and outcome in group psychotherapy: A systematic review

The Influence of Treatment Engagement on Positive Outcomes in the Context of a School-Based Intervention for Students with Externalizing Behavior Problems

The Effects of Self-Monitoring and Performance Feedback on the Treatment Integrity of Behavior Intervention Plan Implementation and Generalization

Intervention in School and Clinic

Trends in Reporting Procedural Integrity: A Comparison

Anticipating Challenges: School-based Social Work Intervention Research

Mental Health Interventions in Middle Schools: A 10-Year Review of Research

A History of Child and Adolescent Treatment Through a Distillation Lens: Looking Back to Move Forward

Toward a Level Playing Field: Predictive Factors for the Outcomes of Mental Health Treatment for Adolescents

Increasing Awareness of Child Mental Health Issues among Elementary School Staff

Measuring Teacher Implementation of the BEST in CLASS Intervention Program and Corollary Child Outcomes

Self-Control Interventions That Benefit Executive Functioning and Academic Outcomes in Early and Middle Childhood

Examining the Role of Student Responsiveness in Treatment Effects of a Tier 2 Program Targeting Reductions in Problem Behavior