Abstract:Issue: The way educators think about the nature of competence, the approaches one selects for the assessment of competence, what generated data implies, and what counts as good assessment now involve broader and more diverse interpretive processes. Broadening philosophical positions in assessment has educators applying different interpretations to similar assessment concepts. As a result, what is claimed through assessment, including what counts as quality, can be different for each of us despite using similar activities and language. This is leading to some uncertainty on how to proceed or worse, provides opportunities for questioning the legitimacy of any assessment activity or outcome. While some debate in assessment is inevitable, most have been within philosophical positions (e.g., how best to minimize error), whereas newer debates are happening across philosophical positions (e.g., whether error is a useful concept). As new ways of approaching assessment have emerged, the interpretive nature of underlying philosophical positions has not been sufficiently attended to. Evidence: We illustrate interpretive processes of assessment in action by: (a) summarizing the current health professions assessment context from a philosophical perspective as a way of describing its evolution; (b) demonstrating implications in practice using two examples (i.e., analysis of assessment work and validity claims); and (c) examining pragmatism to demonstrate how even within specific philosophical positions opportunities for variable interpretations still exist. Implications: Our concern is not that assessment designers and users have different assumptions, but that practically, educators may unknowingly (or insidiously) apply different assumptions, and methodological and interpretive norms, and subsequently settle on different views on what serves as quality assessment even for the same assessment program or event. With the state of assessment in health professions in flux, we conclude by calling for a philosophically explicit approach to assessment, and underscore assessment as, fundamentally, an interpretive process - one which demands the careful elucidation of philosophical assumptions to promote understanding and ultimately defensibility of assessment processes and outcomes.

One size fits all? What counts as quality practice in (reflexive) thematic analysis?

Conceptual and design thinking for thematic analysis

Is thematic analysis used well in health psychology? A critical review of published research, with recommendations for quality practice and reporting

Reflecting on reflexive thematic analysis

A choir or cacophony? Sample sizes and quality of conveying participants’ voices in phenomenological research

Translation quality assessment: a critical methodological review

Attempting rigour and replicability in thematic analysis of qualitative research data; a case study of codebook development

Supporting best practice in reflexive thematic analysis reporting in Palliative Medicine: A review of published research and introduction to the Reflexive Thematic Analysis Reporting Guidelines (RTARG)

To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales

Reflexive Thematic Analysis for Applied Qualitative Health Research

What is a theme? Teaching thematic analysis in qualitative communication research methods

How do language education researchers attend to quality in qualitative studies?

Research and scholarly methods: Thematic analysis

Applying Thematic Analysis to Education: A Hybrid Approach to Interpreting Data in Practitioner Research

The pursuit of quality in grounded theory

Reporting guidelines for qualitative research: a values-based approach

Critical Thinking: Two Theses from the Ground Up

Dilemmas in qualitative health research

Learning to Do Qualitative Data Analysis: A Starting Point

Thematic Analysis with Open-Source Generative AI and Machine Learning: A New Method for Inductive Qualitative Codebook Development

Attending to Variable Interpretations of Assessment Science and Practice