Abstract:Background: Personalized asthma management depends on a clinician's ability to efficiently review patient's data and make timely clinical decisions. Unfortunately, efficient and effective review of these data is impeded by the varied format, location, and workflow of data acquisition, storage, and processing in the electronic health record. While machine learning (ML) and clinical decision support tools are well-positioned as potential solutions, the translation of such frameworks requires that barriers to implementation be addressed in the formative research stages. Objective: We aimed to use a structured user-centered design approach (double-diamond design framework) to (1) qualitatively explore clinicians' experience with the current asthma management system, (2) identify user requirements to improve algorithm explainability and Asthma Guidance and Prediction System prototype, and (3) identify potential barriers to ML-based clinical decision support system use. Methods: At the "discovery" phase, we first shadowed to understand the practice context. Then, semistructured interviews were conducted digitally with 14 clinicians who encountered pediatric asthma patients at 2 outpatient facilities. Participants were asked about their current difficulties in gathering information for patients with pediatric asthma, their expectations of ideal workflows and tools, and suggestions on user-centered interfaces and features. At the "define" phase, a synthesis analysis was conducted to converge key results from interviewees' insights into themes, eventually forming critical "how might we" research questions to guide model development and implementation. Results: We identified user requirements and potential barriers associated with three overarching themes: (1) usability and workflow aspects of the ML system, (2) user expectations and algorithm explainability, and (3) barriers to implementation in context. Even though the responsibilities and workflows vary among different roles, the core asthma-related information and functions they requested were highly cohesive, which allows for a shared information view of the tool. Clinicians hope to perceive the usability of the model with the ability to note patients' high risks and take proactive actions to manage asthma efficiently and effectively. For optimal ML algorithm explainability, requirements included documentation to support the validity of algorithm development and output logic, and a request for increased transparency to build trust and validate how the algorithm arrived at the decision. Acceptability, adoption, and sustainability of the asthma management tool are implementation outcomes that are reliant on the proper design and training as suggested by participants. Conclusions: As part of our comprehensive informatics-based process centered on clinical usability, we approach the problem using a theoretical framework grounded in user experience research leveraging semistructured interviews. Our focus on meeting the needs of the practice with ML technology is emphasized by a user-centered approach to clinician engagement through upstream technology design.

Evaluating automated machine learning platforms for use in healthcare

Clinician checklist for assessing suitability of machine learning applications in healthcare

A clinician's guide to understanding and critically appraising machine learning studies: a checklist for Ruling Out Bias Using Standard Tools in Machine Learning (ROBUST-ML)

Clinical performance of automated machine learning: A systematic review

Automated Machine Learning for Healthcare and Clinical Notes Analysis

Evaluating the Performance of Automated Machine Learning (AutoML) Tools for Heart Disease Diagnosis and Prediction

Automating Construction of Machine Learning Models With Clinical Big Data: Proposal Rationale and Methods

Machine learning in health care and laboratory medicine: General overview of supervised learning and Auto-ML

Evaluation of Machine Learning Algorithms for Pressure Injury Risk Assessment in a Hospital with Limited IT Resources

Health Risk Assessment Using Machine Learning: Systematic Review

Considerations for Quality Control Monitoring of Machine Learning Models in Clinical Practice

Clinical Needs Assessment of a Machine Learning-Based Asthma Management Tool: User-Centered Design Approach

Designing an ML Auditing Criteria Catalog as Starting Point for the Development of a Framework

Review of the AMLAS Methodology for Application in Healthcare

Evaluation of machine learning algorithms for health and wellness applications: A tutorial

Nurses' perceptions of the design, implementation, and adoption of machine learning clinical decision support: A descriptive qualitative study

Machine learning for the diagnosis of acute coronary syndrome using a 12-lead ECG: a systematic review

Factors influencing clinician and patient interaction with machine learning-based risk prediction models: a systematic review

Inclusion of Clinicians in the Development and Evaluation of Clinical Artificial Intelligence Tools: A Systematic Literature Review

Machine learning for subtype definition and risk prediction in heart failure, acute coronary syndromes and atrial fibrillation: systematic review of validity and clinical utility