Abstract:Background: In both routine practice contexts and research studies, evidence from standardized self-report symptom measures, administered pre- and post-treatment, is predominantly used to determine whether psychotherapy has been successful. Understanding the nature of unsuccessful psychotherapy requires an ability to evaluate the credibility of outcome data generated by such techniques. An important body of research has identified discrepancies between outcomes assessed through symptom measures and those obtained from other sources. However, not enough is known about the extent to which such paradoxical outcomes exist. Objective: This study analyzes the relationship between outcomes, as assessed by a standardized self-report measure, and as assessed by ratings of young people's descriptions of change at post-counseling interviews. Methods: Participants were 50 young people (13–16 years old) who had taken part in a trial of up to 10 weeks of school-based humanistic counseling. Our primary standardized measure was the Young Person's CORE (YP-CORE). To assess young people's experiences of counseling change, three independent raters scrutinized transcripts of post-counseling interviews, and scored levels of helpfulness on a 1 (Not at all helpful) to 10 (Extremely helpful) scale. Inter-rater reliabilities were 0.94 (Cronbach's Alpha) and 0.96 (McDonald's Omega). Sensitivity analyses were conducted to explore relationships between helpfulness ratings and other outcome measures, i.e., satisfaction with counseling (ESQ) and the Goal-Based-Outcome Tool (GBO), and process measures, i.e., the Working Alliance Inventory (WAI-S) and the Barret Lennard Relationship Inventory (BLRI). Results: Multilevel analysis indicated that helpfulness ratings were not significantly associated with changes in YP-CORE scores. Analyzed categorically, 38% of those showing reliable improvement on the standardized measure were below the median for self-described helpfulness, and 47% of those not showing reliable change were at or above the median for self-described helpfulness. Sensitivity analyses demonstrated closer correlations between helpfulness ratings and other outcome measures (ESQ and GBO), and between helpfulness ratings and process measures (WAI-S and BLRI). Discussion: Our results raise questions about reliance on symptom change outcome measures for defining treatment success and failure, given their disparity with clients' own descriptions of the helpfulness of therapy. Implications for practice and research are discussed.

Bridging the feedback implementation gap: A comparison of empirical and rational decision rules in naturalistic psychotherapy

Prospective evaluation of a clinical decision support system in psychological therapy.

The effect of feedback-informed cognitive behavioral therapy on treatment outcome: A randomized controlled trial.

Enhancing the effect of psychotherapy through systematic client feedback in outpatient mental healthcare: A cluster randomized trial.

Using Progress Feedback to Enhance Treatment Outcomes: A Narrative Review

A comparison of scalable routine clinical materials and observer ratings to assess CBT fidelity

Measuring the individualization potential of treatment individualization rules: Application to rules built with a new parametric interaction model for parallel-group clinical trials

Agreement between therapists, parents, patients, and independent evaluators on clinical improvement in pediatric obsessive-compulsive disorder.

Patient perspectives on systematic client feedback in Dutch outpatient mental healthcare, a qualitative case reports study

Toward a Level Playing Field: Predictive Factors for the Outcomes of Mental Health Treatment for Adolescents

Should We Use Clinician Self-Report to Tailor Implementation Strategies? Predicting Use of Youth CBT with Clinician Self-Report Versus Direct Observation

Putting measurement-based care into action: A mixed methods study of the benefits of integrating routine client feedback in coordinated specialty care programs for early psychosis

Responsive regulation of treatment duration in routine practice in United Kingdom primary care settings: Replication in a larger sample.

Feedback on clinical team performance: how does it work, in what contexts, for whom, and for what changes? A critical realist qualitative multiple case study

When meta-analysis misleads: A critical case study of a meta-analysis of client feedback.

Beyond outcomes monitoring: measurement feedback systems in child and adolescent clinical practice

Dose-effect relations and responsive regulation of treatment duration: The good enough level.

How do we know whether treatment has failed? Paradoxical outcomes in counseling with young people

Does outcome feedback make you a better emergency physician? A systematic review and research framework proposal

Performance and Application of Estimators for the Value of an Optimal Dynamic Treatment Rule

Predicting treatment failure in regular care Internet-Delivered Cognitive Behavior Therapy for depression and anxiety using only weekly symptom measures.