Abstract:To decrease medication errors, a common cause of injury to patients, we developed a novel electronic tool to facilitate asynchronous communication between healthcare professionals (HCPs) and patients for medication reconciliation. However, it was unknown whether the tool adequately supported HCPs’ usability needs. Our objective was to conduct an iterative usability evaluation of the tool with physicians, nurses, and pharmacists, in preparation for a randomized controlled trial. We hypothesized that we would identify design weaknesses that could be addressed via interface modifications prior to the trial. We completed a mixed-method, formative usability evaluation with 20 HCPs in the Veterans Affairs (VA) Health Services Research and Development, Human-Computer Interaction and Simulation Laboratory located within a major medical center. The tool in this study is formally known as the Secure Messaging for Medication Reconciliation Tool (SMMRT). The evaluation consisted of four sequential steps: 1) phase I usability testing to assess the baseline tool along with small, iterative design changes throughout testing; 2) heuristic evaluation; 3) implement major design changes that incorporate findings from previous steps; and 4) phase II usability testing to assess the implemented design changes and further refine the tool. This presentation focuses on steps 1 and 4 related to usability testing. During testing, HCPs worked through a real case consisting of a patient discharged from the hospital within the past 30 days who had at least 5 outpatient medications. We collected data on efficiency, usability errors, and participants’ satisfaction, along with participants’ ability to detect and address three distinct types of medication errors via the tool. For the latter, we inserted three safety probes into the simulation: 1) a missing medication (i.e., omission); 2) an extraneous medication (i.e., commission); and 3) an inaccurate dose (i.e., dose discrepancy). Data were analyzed descriptively, rather than via statistical comparisons, due to the formative and iterative nature of this research. There was no indication of efficiency gains during iterative prototyping and testing. Highlights of usability errors included confusion about medication entry fields; incorrect assumptions regarding medication list accuracy; inadequate medication information sorting and organization; and premature closure. Additionally, HCPs described usability errors that might occur in clinical practice. For example, medication images on the tool may not match what is dispensed to patients. HCPs also expressed concern that medication updates made via the tool may not be consistently updated in the electronic health record. In terms of satisfaction, HCPs’ ratings tended to increase as design modifications were implemented. After phase II usability testing, their overall satisfaction was favorable. Finally, for each of the three safety probes, 50% or fewer of HCPs identified the associated medication error. This research illustrates the importance of usability evaluations as a precursor to randomized trials of health information technology. Our multi-step approach to usability testing, with heuristic evaluation at the midpoint, may inform the design of other usability evaluations. While efficiency gains were not realized, user satisfaction improved. The inclusion of safety probes was especially valuable, since probes allowed us to assess error detection rates. There may be opportunities for human factors professionals to expand the sophistication and types of probes used in future healthcare research. Future studies are needed to develop more advanced design approaches that facilitate healthcare professionals’ detection of medication errors.

Clinical research staff perceptions on a natural language processing-driven tool for eligibility prescreening: An iterative usability assessment

"The Truth Is, We Must Miss Some": A Qualitative Study of the Patient Eligibility Screening Process, and Automation Perspectives, for Cancer Clinical Trials

A usability evaluation of the perceived user friendliness, accessibility, and inclusiveness of a personalized digital care pathway tool

Evaluating the Impact on Clinical Task Efficiency of a Natural Language Processing Algorithm for Searching Medical Documents: Prospective Crossover Study

Formative Usability Evaluation of a Novel Tool for Medication Reconciliation

Criteria2Query: a natural language interface to clinical databases for cohort definition

A Patient-Screening Tool for Clinical Research Based on Electronic Health Records Using OpenEHR: Development Study

Improving the Efficacy of the Data Entry Process for Clinical Research with a Natural Language Processing–Driven Medical Information Extraction System: Quantitative Field Research

A query interface for clinical research with Chinese electronic health record using Natural Language Processing

ClinicalTrials.gov as a data source for semi-automated point-of-care trial eligibility screening

Automatic trial eligibility surveillance based on unstructured clinical data

Evaluation of Three Feasibility Tools for Identifying Patient Data and Biospecimen Availability: Comparative Usability Study

User Requirements for an Electronic Patient Recruitment System: Semistructured Interview Analysis After First Implementation in 3 German University Hospitals

Automated clinical trial eligibility prescreening: increasing the efficiency of patient identification for clinical trials in the emergency department

Evaluating the usability of a co-designed interactive web application for cardiac rehabilitation.

Usability Testing of Situation Awareness Clinical Decision Support in the ICU

Leveraging Real-World Data for the Selection of Relevant Eligibility Criteria for the Implementation of Electronic Recruitment Support in Clinical Trials

Applying cognitive walkthrough methodology to improve the usability of an equity-focused implementation strategy

The Development of a Digital Patient Navigation Tool to Increase Colorectal Cancer Screening Among Federally Qualified Health Center Patients: Acceptability and Usability Testing

Can a computerized format replace a paper form in PRO and HRQL evaluation? Psychometric testing of the computer-assisted LCSS instrument (eLCSS-QL)

Formative Usability Evaluation of a Three-Way Digital Healthcare System for the People with Disabilities and Their Caregivers: A Cross-Sectional Study