Abstract:Purpose: Usability testing can be used to evaluate human-computer interaction (HCI) and communication in shared decision making (SDM) for patient-provider behavioral change and behavioral contracting. Traditional evaluations of usability using scripted or mock patient scenarios with think-aloud protocol analysis provide a way to identify HCI issues. In this paper we describe the application of these methods in the evaluation of the Avoiding Diabetes Thru Action Plan Targeting (ADAPT) tool, and test the usability of the tool to support the ADAPT framework for integrated care counseling of pre-diabetes. The think-aloud protocol analysis typically does not provide an assessment of how patient-provider interactions are effected in "live" clinical workflow or whether a tool is successful. Therefore, "Near-live" clinical simulations involving applied simulation methods were used to compliment the think-aloud results. This complementary usability technique was used to test the end-user HCI and tool performance by more closely mimicking the clinical workflow and capturing interaction sequences along with assessing the functionality of computer module prototypes on clinician workflow. We expected this method to further complement and provide different usability findings as compared to think-aloud analysis. Together, this mixed method evaluation provided comprehensive and realistic feedback for iterative refinement of the ADAPT system prior to implementation. Methods: The study employed two phases of testing of a new interactive ADAPT tool that embedded an evidence-based shared goal setting component into primary care workflow for dealing with pre-diabetes counseling within a commercial physician office electronic health record (EHR). Phase I applied usability testing that involved "think-aloud" protocol analysis of eight primary care providers interacting with several scripted clinical scenarios. Phase II used "near-live" clinical simulations of five providers interacting with standardized trained patient actors enacting the clinical scenario of counseling for pre-diabetes, each of whom had a pedometer that recorded the number of steps taken over a week. In both phases, all sessions were audio-taped and motion screen-capture software was activated for onscreen recordings. Transcripts were coded using iterative qualitative content analysis methods. Results: In Phase I, the impact of the components and layout of ADAPT on user's Navigation, Understandability, and Workflow were associated with the largest volume of negative comments (i.e. approximately 80% of end-user commentary), while Usability and Content of ADAPT were representative of more positive than negative user commentary. The heuristic category of Usability had a positive-to-negative comment ratio of 2.1, reflecting positive perception of the usability of the tool, its functionality, and overall co-productive utilization of ADAPT. However, there were mixed perceptions about content (i.e., how the information was displayed, organized and described in the tool). In Phase II, the duration of patient encounters was approximately 10 min with all of the Patient Instructions (prescriptions) and behavioral contracting being activated at the end of each visit. Upon activation, providers accepted the pathway prescribed by the tool 100% of the time and completed all the fields in the tool in the simulation cases. Only 14% of encounter time was spent using the functionality of the ADAPT tool in terms of keystrokes and entering relevant data. The rest of the time was spent on communication and dialog to populate the patient instructions. In all cases, the interaction sequence of reviewing and discussing exercise and diet of the patient was linked to the functionality of the ADAPT tool in terms of monitoring, response-efficacy, self-efficacy, and negotiation in the patient-provider dialog. There was a change from one-way dialog to two-way dialog and negotiation that ended in a behavioral contract. This change demonstrated the tool's sequence, which supported recording current exercise and diet followed by a diet and exercise goal setting procedure to reduce the risk of diabetes onset. Conclusions: This study demonstrated that "think-aloud" protocol analysis with "near-live" clinical simulations provided a successful usability evaluation of a new primary care pre-diabetes shared goal setting tool. Each phase of the study provided complementary observations on problems with the new onscreen tool and was used to show the influence of the ADAPT framework on the usability, workflow integration, and communication between the patient and provider. The think-aloud tests with the provider showed the tool can be used according to the ADAPT framework (exercise-to-diet behavior change and tool utilization), while the clinical simulations revealed the ADAPT framework to realistically support patient-provider communication to obtain behavioral change contract. SDM interactions and mechanisms affecting protocol-based care can be more completely captured by combining "near-live" clinical simulations with traditional "think-aloud analysis" which augments clinician utilization. More analysis is required to verify if the rich communication actions found in Phase II compliment clinical workflows.

Shared decision-making and detection of comorbidities in an online acromegaly consultation with and without the Acromegaly Disease Activity Tool ACRODAT® using the simulated person approach

Questionnaire and tools: clinical powerful instrument in acromegaly diagnosis and management

Preferences for different treatment options among people living with acromegaly in the US

SAT622 Matched Physician- and Patient-reported Responses of Acromegaly Symptom Control: Comparisons from the Macro Registry

Endogenous filamentous fungal endophthalmitis – single‐centre survey in patients with acute leukaemia or postallogeneic stem cell transplantation and review of the literature

Approach to the patient with controlled acromegaly and acromegalic arthropathy: clinical diagnosis and management

Investigating shared decision-making during the use of a digital health tool for physical activity planning in cardiac rehabilitation

Prospectively investigating the impact of AI onshared decision-making in post kidney transplant care (PRIMA-AI): protocol for a longitudinal qualitative study among patients, their support persons and treating physicians at a tertiary care centre

Personalized medicine in acromegaly: The ACROFAST study

Exploring potential influencing factors of inadherence to specialist aftercare and long-term medication in patients with acromegaly

Beyond pros and cons - developing a patient decision aid to cultivate dialog to build relationships: insights from a qualitative study and decision aid development

Evaluation of the impact Of ChatGPT support on acromegaly management and patient education

Machine learning-based algorithms applied to drug prescriptions and other healthcare services in the Sicilian claims database to identify acromegaly as a model for the earlier diagnosis of rare diseases

Two surgeons for one pancreas

Digital voice analysis as a biomarker of acromegaly

Evaluation of a hybrid telehealth care pathway for patients with axial spondyloarthritis including self-sampling at home: results of a longitudinal proof-of-concept mixed-methods study (TeleSpactive)

[Effect of the anterior pituitary on the islands of Langerhans in the guinea pig].

Utilising Adaptive Choice‐Based Conjoint Approach to Facilitate Shared Decision‐Making in Osteoarthritis Management: A Patient Perception Study

ACROMORFO study: gait analysis in a cohort of acromegalic patients

Usability testing of Avoiding Diabetes Thru Action Plan Targeting (ADAPT) decision support for integrating care-based counseling of pre-diabetes in an electronic health record

New rearranged chromosomes t(2;3), t(7;14), t(8;11), and t(7;12;14), in a case of T-cell chronic lymphocytic leukemia.