Abstract:Introduction: Clinical reasoning (CR) is a complex skill enabling transition from clinical novice to expert decision maker. The Objective Structured Clinical Examination (OSCE) is widely used to evaluate clinical competency, though there is limited literature exploring how this assessment is best used to assess CR skills. This proof-of-concept study explored the creation and pilot testing of a post-station CR assessment, named Oral Debrief (OD), in the context of undergraduate medical education. Methods: A modified-Delphi technique was used to create a standardised domain-based OD marking rubric encapsulating the key skills of CR that drew upon existing literature and our existing placement-based CR tool. 16 OSCE examiners were recruited to score three simulated OD recordings that were scripted to portray differing levels of competency. Adopting a think-aloud approach, examiners vocalised their thought processes while utilising the rubric to assess each video. Thereafter, semi-structured interviews explored examiners' views on the OD approach. Recordings were transcribed, anonymised and analysed deductively and inductively for recurring themes. Additionally, inter-rater agreement of examiners' scoring was determined using the Fleiss Kappa statistic both within group and in comparison to a reference examiner group. Results: The rubric achieved fair to good levels of inter-rater reliability metrics across its constituent domains and overall global judgement scales. Think-aloud scoring revealed that participating examiners considered several factors when scoring students' CR abilities. This included the adoption of a confident structured approach, discriminating between relevant and less-relevant information, and the ability to prioritise and justify decision making. Furthermore, students' CR skills were judged in light of potential risks to patient safety and examiners' own illness scripts. Feedback from examiners indicated that whilst additional training in rubric usage would be beneficial, OD offered a positive approach for examining CR ability. Conclusion: This pilot study has demonstrated promising results for the use of a novel post-station OD task to evaluate medical students' CR ability in the OSCE setting. Further work is now planned to evaluate how the OD approach can most effectively be implemented into routine assessment practice.

Psychometric validation of the reconstructed version of the assessment of reasoning tool

Clinical reasoning performance assessment: using situated cognition theory as a conceptual framework

Preliminary Validation of a Clinical Reasoning Theory-Based Assessment Rubric: An e-Delphi Study

Evaluating Clinical Reasoning in Undergraduate Medical Education: The Value of a Virtual Oral Assessment

The script concordance test as a measure of clinical reasoning: a national validation study

Measuring Rational Thinking in Adolescents: The Assessment of Rational Thinking for Youth (ART‐Y)

Development and Validation of a Formative Assessment Tool for Nephrology Fellows' Clinical Reasoning

Validity and reliability of Brier scoring for assessment of probabilistic diagnostic reasoning

Assessing clinical reasoning in undergraduate medical students during history taking with an empirically derived scale for clinical reasoning indicators

A conditional reasoning test for risk and incident propensity: Development and validation.

Measuring deliberate reflection in residents: validation and psychometric properties of a measurement tool

Evaluation of the reliability of diagnostic thinking inventory for physical therapist and of the development of clinical reasoning in physiotherapy students

Assessing clinical reasoning in the OSCE: pilot-testing a novel oral debrief exercise

Development and Psychometric Testing of the Clinical Reasoning Scale Among Nursing Students Enrolled in Three Types of Programs in Taiwan

The influence of a digital clinical reasoning test on medical student learning behavior during clinical clerkships

Validation of Metacognitive Awareness Inventory from a Private Medical University in India

Research Competencies Assessment Instrument for Nurses: Preliminary Psychometric Properties

Psychometric Evaluation of the Nurse Clinical Reasoning Scale: A Validation Study Among Italian Nurses

Development of Analytic Rubric for Assessing Written Assignments for Evaluating Higher Order Cognition in the First Phase Neuroanatomy Module

Implementing a Critical Thinking Tool to Evaluate Educational Needs for Inpatient Rehabilitation Nurses