Simplifying proximal isovelocity surface area as an assessment method of mitral valve area in patients with rheumatic mitral stenosis by fixing aliasing velocity and mitral valve angle.

Alaa Mabrouk Salem Omar,M. A. Abdel-Rahman,Hidekazu Tanaka,O. Rifaie

DOI: https://doi.org/10.1016/j.jsha.2012.11.004

Journal of the Saudi Heart Association

Abstract:

What problem does this paper attempt to address?

Toward Successful Implementation of Artificial Intelligence in Health Care Practice: Protocol for a Research Program

Petra Svedberg,Julie Reed,Per Nilsen,James Barlow,Carl Macrae,Jens Nygren

DOI: https://doi.org/10.2196/34920

2022-03-09

Abstract:Background: The uptake of artificial intelligence (AI) in health care is at an early stage. Recent studies have shown a lack of AI-specific implementation theories, models, or frameworks that could provide guidance for how to translate the potential of AI into daily health care practices. This protocol provides an outline for the first 5 years of a research program seeking to address this knowledge-practice gap through collaboration and co-design between researchers, health care professionals, patients, and industry stakeholders. Objective: The first part of the program focuses on two specific objectives. The first objective is to develop a theoretically informed framework for AI implementation in health care that can be applied to facilitate such implementation in routine health care practice. The second objective is to carry out empirical AI implementation studies, guided by the framework for AI implementation, and to generate learning for enhanced knowledge and operational insights to guide further refinement of the framework. The second part of the program addresses a third objective, which is to apply the developed framework in clinical practice in order to develop regional capacity to provide the practical resources, competencies, and organizational structure required for AI implementation; however, this objective is beyond the scope of this protocol. Methods: This research program will use a logic model to structure the development of a methodological framework for planning and evaluating implementation of AI systems in health care and to support capacity building for its use in practice. The logic model is divided into time-separated stages, with a focus on theory-driven and coproduced framework development. The activities are based on both knowledge development, using existing theory and literature reviews, and method development by means of co-design and empirical investigations. The activities will involve researchers, health care professionals, and other stakeholders to create a multi-perspective understanding. Results: The project started on July 1, 2021, with the Stage 1 activities, including model overview, literature reviews, stakeholder mapping, and impact cases; we will then proceed with Stage 2 activities. Stage 1 and 2 activities will continue until June 30, 2026. Conclusions: There is a need to advance theory and empirical evidence on the implementation requirements of AI systems in health care, as well as an opportunity to bring together insights from research on the development, introduction, and evaluation of AI systems and existing knowledge from implementation research literature. Therefore, with this research program, we intend to build an understanding, using both theoretical and empirical approaches, of how the implementation of AI systems should be approached in order to increase the likelihood of successful and widespread application in clinical practice. International registered report identifier (irrid): PRR1-10.2196/34920.
A comprehensive overview of barriers and strategies for AI implementation in healthcare: Mixed-method design

Monika Nair,Petra Svedberg,Ingrid Larsson,Jens M Nygren

DOI: https://doi.org/10.1371/journal.pone.0305949

IF: 3.7

2024-08-09

PLoS ONE

Abstract:Implementation of artificial intelligence systems for healthcare is challenging. Understanding the barriers and implementation strategies can impact their adoption and allows for better anticipation and planning. This study's objective was to create a detailed inventory of barriers to and strategies for AI implementation in healthcare to support advancements in methods and implementation processes in healthcare. A sequential explanatory mixed method design was used. Firstly, scoping reviews and systematic literature reviews were identified using PubMed. Selected studies included empirical cases of AI implementation and use in clinical practice. As the reviews were deemed insufficient to fulfil the aim of the study, data collection shifted to the primary studies included in those reviews. The primary studies were screened by title and abstract, and thereafter read in full text. Then, data on barriers to and strategies for AI implementation were extracted from the included articles, thematically coded by inductive analysis, and summarized. Subsequently, a direct qualitative content analysis of 69 interviews with healthcare leaders and healthcare professionals confirmed and added results from the literature review. Thirty-eight empirical cases from the six identified scoping and literature reviews met the inclusion and exclusion criteria. Barriers to and strategies for AI implementation were grouped under three phases of implementation (planning, implementing, and sustaining the use) and were categorized into eleven concepts; Leadership, Buy-in, Change management, Engagement, Workflow, Finance and human resources, Legal, Training, Data, Evaluation and monitoring, Maintenance. Ethics emerged as a twelfth concept through qualitative analysis of the interviews. This study illustrates the inherent challenges and useful strategies in implementing AI in healthcare practice. Future research should explore various aspects of leadership, collaboration and contracts among key stakeholders, legal strategies surrounding clinicians' liability, solutions to ethical dilemmas, infrastructure for efficient integration of AI in workflows, and define decision points in the implementation process.
Artificial Intelligence Applications in Health Care Practice: Scoping Review

Malvika Sharma,Carl Savage,Monika Nair,Ingrid Larsson,Petra Svedberg,Jens M Nygren

DOI: https://doi.org/10.2196/40238

IF: 7.076

2022-10-06

Journal of Medical Internet Research

Abstract:Background: Artificial intelligence (AI) is often heralded as a potential disruptor that will transform the practice of medicine. The amount of data collected and available in health care, coupled with advances in computational power, has contributed to advances in AI and an exponential growth of publications. However, the development of AI applications does not guarantee their adoption into routine practice. There is a risk that despite the resources invested, benefits for patients, staff, and society will not be realized if AI implementation is not better understood. Objective: The aim of this study was to explore how the implementation of AI in health care practice has been described and researched in the literature by answering 3 questions: What are the characteristics of research on implementation of AI in practice? What types and applications of AI systems are described? What characteristics of the implementation process for AI systems are discernible? Methods: A scoping review was conducted of MEDLINE (PubMed), Scopus, Web of Science, CINAHL, and PsycINFO databases to identify empirical studies of AI implementation in health care since 2011, in addition to snowball sampling of selected reference lists. Using Rayyan software, we screened titles and abstracts and selected full-text articles. Data from the included articles were charted and summarized. Results: Of the 9218 records retrieved, 45 (0.49%) articles were included. The articles cover diverse clinical settings and disciplines; most (32/45, 71%) were published recently, were from high-income countries (33/45, 73%), and were intended for care providers (25/45, 56%). AI systems are predominantly intended for clinical care, particularly clinical care pertaining to patient-provider encounters. More than half (24/45, 53%) possess no action autonomy but rather support human decision-making. The focus of most research was on establishing the effectiveness of interventions (16/45, 35%) or related to technical and computational aspects of AI systems (11/45, 24%). Focus on the specifics of implementation processes does not yet seem to be a priority in research, and the use of frameworks to guide implementation is rare. Conclusions: Our current empirical knowledge derives from implementations of AI systems with low action autonomy and approaches common to implementations of other types of information systems. To develop a specific and empirically based implementation framework, further research is needed on the more disruptive types of AI systems being implemented in routine care and on aspects unique to AI implementation in health care, such as building trust, addressing transparency issues, developing explainable and interpretable solutions, and addressing ethical concerns around privacy and data protection.

health care sciences & services,medical informatics
From Bit To Bedside: A Practical Framework For Artificial Intelligence Product Development In Healthcare

David Higgins,Vince I. Madai

DOI: https://doi.org/10.1002/aisy.202000052

2020-03-23

Abstract:Artificial Intelligence (AI) in healthcare holds great potential to expand access to high-quality medical care, whilst reducing overall systemic costs. Despite hitting the headlines regularly and many publications of proofs-of-concept, certified products are failing to breakthrough to the clinic. AI in healthcare is a multi-party process with deep knowledge required in multiple individual domains. The lack of understanding of the specific challenges in the domain is, therefore, the major contributor to the failure to deliver on the big promises. Thus, we present a decision perspective framework, for the development of AI-driven biomedical products, from conception to market launch. Our framework highlights the risks, objectives and key results which are typically required to proceed through a three-phase process to the market launch of a validated medical AI product. We focus on issues related to Clinical validation, Regulatory affairs, Data strategy and Algorithmic development. The development process we propose for AI in healthcare software strongly diverges from modern consumer software development processes. We highlight the key time points to guide founders, investors and key stakeholders throughout their relevant part of the process. Our framework should be seen as a template for innovation frameworks, which can be used to coordinate team communications and responsibilities towards a reasonable product development roadmap, thus unlocking the potential of AI in medicine.

Computers and Society,Artificial Intelligence,Human-Computer Interaction,Applications,Machine Learning
ModelEd, TestEd, TrustEd

J. Katoen,R. Langerak,A. Rensink

DOI: https://doi.org/10.1007/978-3-319-68270-9

Abstract:
An ethics assessment tool for artificial intelligence implementation in healthcare: CARE-AI

Yilin Ning,Xiaoxuan Liu,Gary S. Collins,Karel G. M. Moons,Melissa McCradden,Daniel Shu Wei Ting,Jasmine Chiat Ling Ong,Benjamin Alan Goldstein,Siegfried K. Wagner,Pearse A. Keane,Eric J. Topol,Nan Liu

DOI: https://doi.org/10.1038/s41591-024-03310-1

IF: 82.9

2024-10-20

Nature Medicine

Abstract:The deployment of artificial intelligence (AI)-powered prediction models in healthcare can lead to ethical concerns about their implementation and upscaling. For example, AI prediction models can hinder clinical decision-making if they advise different diagnoses or treatments by sex and gender or by race and ethnicity without clear justification. Recent guidance (such as the WHO guidance on ethics and governance of AI for health and the Dutch guideline on AI for healthcare) and legislation (such as the European Union AI Act and the White House Executive Order on Safe, Secure, and Trustworthy Development and Use of AI in United States) have outlined important principles for the implementation of AI, including ethical considerations 1,2 . Health systems have responded by establishing governance committees and processes to ensure the safe and equitable implementation of AI tools 3 . However, there is currently no assessment tool that can identify and mitigate ethical issues during the implementation of AI prediction models in healthcare practice, including for public health. The development and validation of AI prediction models has benefited from detailed reporting and risk-of-bias tools, such as TRIPOD+AI 4 and PROBAST (with its forthcoming AI extension) for fairness and bias control and CLAIM 5 for data privacy, security and interpretability of AI imaging studies. However, when planning the implementation of a rigorously developed and well-performing AI prediction model in healthcare practice, existing recommendations and guidance on ethics are sparse and lack operational detail. For example, the DECIDE-AI reporting guideline 6 contains a small number of ethics-related recommendations for early clinical evaluation of AI concerning equity, safety and human-AI interaction, and FUTURE-AI 7 provides recommendations based on six principles (fairness, universality, traceability, usability, robustness and explainability) in model design, development, validation and deployment. A bioethics-centric delivery science toolkit for responsible AI implementation in healthcare is needed 8 .

biochemistry & molecular biology,cell biology,medicine, research & experimental
Human Decision-making in an Artificial Intelligence-Driven Future in Health: Protocol for Comparative Analysis and Simulation

Nandini Doreswamy,Louise Horstmanshof

DOI: https://doi.org/10.2196/42353

2022-12-23

Abstract:Background: Health care can broadly be divided into two domains: clinical health services and complex health services (ie, nonclinical health services, eg, health policy and health regulation). Artificial intelligence (AI) is transforming both of these areas. Currently, humans are leaders, managers, and decision makers in complex health services. However, with the rise of AI, the time has come to ask whether humans will continue to have meaningful decision-making roles in this domain. Further, rationality has long dominated this space. What role will intuition play? Objective: The aim is to establish a protocol of protocols to be used in the proposed research, which aims to explore whether humans will continue in meaningful decision-making roles in complex health services in an AI-driven future. Methods: This paper describes a set of protocols for the proposed research, which is designed as a 4-step project across two phases. This paper describes the protocols for each step. The first step is a scoping review to identify and map human attributes that influence decision-making in complex health services. The research question focuses on the attributes that influence human decision-making in this context as reported in the literature. The second step is a scoping review to identify and map AI attributes that influence decision-making in complex health services. The research question focuses on attributes that influence AI decision-making in this context as reported in the literature. The third step is a comparative analysis: a narrative comparison followed by a mathematical comparison of the two sets of attributes-human and AI. This analysis will investigate whether humans have one or more unique attributes that could influence decision-making for the better. The fourth step is a simulation of a nonclinical environment in health regulation and policy into which virtual human and AI decision makers (agents) are introduced. The virtual human and AI will be based on the human and AI attributes identified in the scoping reviews. The simulation will explore, observe, and document how humans interact with AI, and whether humans are likely to compete, cooperate, or converge with AI. Results: The results will be presented in tabular form, visually intuitive formats, and-in the case of the simulation-multimedia formats. Conclusions: This paper provides a road map for the proposed research. It also provides an example of a protocol of protocols for methods used in complex health research. While there are established guidelines for a priori protocols for scoping reviews, there is a paucity of guidance on establishing a protocol of protocols. This paper takes the first step toward building a scaffolding for future guidelines in this regard. International registered report identifier (irrid): PRR1-10.2196/42353.
Creation of an Evidence-Based Implementation Framework for Digital Health Technology in the Intensive Care Unit: Qualitative Study

Lina Katharina Mosch,Akira-Sebastian Poncette,Claudia Spies,Steffen Weber-Carstens,Monique Schieler,Henning Krampe,Felix Balzer

DOI: https://doi.org/10.2196/22866

2022-04-08

Abstract:Background: Digital health technologies such as continuous remote monitoring and artificial intelligence-driven clinical decision support systems could improve clinical outcomes in intensive care medicine. However, comprehensive evidence and guidelines for the successful implementation of digital health technologies into specific clinical settings such as the intensive care unit (ICU) are scarce. We evaluated the implementation of a remote patient monitoring platform and derived a framework proposal for the implementation of digital health technology in an ICU. Objective: This study aims to investigate barriers and facilitators to the implementation of a remote patient monitoring technology and to develop a proposal for an implementation framework for digital health technology in the ICU. Methods: This study was conducted from May 2018 to March 2020 during the implementation of a tablet computer-based remote patient monitoring system. The system was installed in the ICU of a large German university hospital as a supplementary monitoring device. Following a hybrid qualitative approach with inductive and deductive elements, we used the Consolidated Framework for Implementation Research and the Expert Recommendations for Implementing Change to analyze the transcripts of 7 semistructured interviews with clinical ICU stakeholders and descriptive questionnaire data. The results of the qualitative analysis, together with the findings from informal meetings, field observations, and previous explorations, provided the basis for the derivation of the proposed framework. Results: This study revealed an insufficient implementation process due to lack of staff engagement and few perceived benefits from the novel solution. Further implementation barriers were the high staff presence and monitoring coverage in the ICU. The implementation framework includes strategies to be applied before and during implementation, targeting the implementation setting by involving all ICU stakeholders, assessing the intervention's adaptability, facilitating the implementation process, and maintaining a vital feedback culture. Setting up a unit responsible for implementation, considering the guidance of an implementation advisor, and building on existing institutional capacities could improve the institutional context of implementation projects in the ICU. Conclusions: Implementation of digital health in the ICU should involve a thorough preimplementation assessment of the ICU's need for innovation and its readiness to change, as well as an ongoing evaluation of the implementation conditions. Involvement of all stakeholders, transparent communication, and continuous feedback in an equal atmosphere are essential, but leadership roles must be clearly defined and competently filled. Our proposed framework may guide health care providers with concrete, evidence-based, and step-by-step recommendations for implementation practice, facilitating the introduction of digital health in intensive care. Trial registration: ClinicalTrials.gov NCT03514173; https://clinicaltrials.gov/ct2/show/NCT03514173.
The Clinical Practice Integration of Artificial Intelligence (CPI-AI) framework

Luke Farrow,Dominic Meek,Georgios Leontidis,Marion Campbell,Ewen Harrison,Lesley Anderson

DOI: https://doi.org/10.1302/2046-3758.139.BJR-2024-0135.R1

2024-09-18

Abstract:Despite the vast quantities of published artificial intelligence (AI) algorithms that target trauma and orthopaedic applications, very few progress to inform clinical practice. One key reason for this is the lack of a clear pathway from development to deployment. In order to assist with this process, we have developed the Clinical Practice Integration of Artificial Intelligence (CPI-AI) framework - a five-stage approach to the clinical practice adoption of AI in the setting of trauma and orthopaedics, based on the IDEAL principles (https://www.ideal-collaboration.net/). Adherence to the framework would provide a robust evidence-based mechanism for developing trust in AI applications, where the underlying algorithms are unlikely to be fully understood by clinical teams.
The 7-stranded structure of relaxed scallop muscle myosin filaments: support for a common head configuration in myosin-regulated muscles.

Hind A. Al-Khayat,E. Morris,J. Squire

DOI: https://doi.org/10.1016/j.jsb.2009.02.006

IF: 3.234

2009-05-01

Journal of Structural Biology

Abstract:
Evaluating Artificial Intelligence in Clinical Settings-Let Us Not Reinvent the Wheel

Kathrin Cresswell,Nicolette de Keizer,Farah Magrabi,Robin Williams,Michael Rigby,Mirela Prgomet,Polina Kukhareva,Zoie Shui-Yee Wong,Philip Scott,Catherine K Craven,Andrew Georgiou,Stephanie Medlock,Jytte Brender McNair,Elske Ammenwerth

DOI: https://doi.org/10.2196/46407

2024-08-07

Abstract:Given the requirement to minimize the risks and maximize the benefits of technology applications in health care provision, there is an urgent need to incorporate theory-informed health IT (HIT) evaluation frameworks into existing and emerging guidelines for the evaluation of artificial intelligence (AI). Such frameworks can help developers, implementers, and strategic decision makers to build on experience and the existing empirical evidence base. We provide a pragmatic conceptual overview of selected concrete examples of how existing theory-informed HIT evaluation frameworks may be used to inform the safe development and implementation of AI in health care settings. The list is not exhaustive and is intended to illustrate applications in line with various stakeholder requirements. Existing HIT evaluation frameworks can help to inform AI-based development and implementation by supporting developers and strategic decision makers in considering relevant technology, user, and organizational dimensions. This can facilitate the design of technologies, their implementation in user and organizational settings, and the sustainability and scalability of technologies.
Barriers and facilitators to implementing imaging-based diagnostic artificial intelligence-assisted decision-making software in hospitals in China: a qualitative study using the updated Consolidated Framework for Implementation Research

Liao,X.,Yao,C.,Jin,F.,Zhang,J.,Liu,L.

DOI: https://doi.org/10.1136/bmjopen-2024-084398

IF: 3.006

2024-09-12

BMJ Open

Abstract:Objectives To identify the barriers and facilitators to the successful implementation of imaging-based diagnostic artificial intelligence (AI)-assisted decision-making software in China, using the updated Consolidated Framework for Implementation Research (CFIR) as a theoretical basis to develop strategies that promote effective implementation. Design This qualitative study involved semistructured interviews with key stakeholders from both clinical settings and industry. Interview guide development, coding, analysis and reporting of findings were thoroughly informed by the updated CFIR. Setting Four healthcare institutions in Beijing and Shanghai and two vendors of AI-assisted decision-making software for lung nodules detection and diabetic retinopathy screening were selected based on purposive sampling. Participants A total of 23 healthcare practitioners, 6 hospital informatics specialists, 4 hospital administrators and 7 vendors of the selected AI-assisted decision-making software were included in the study. Results Within the 5 CFIR domains, 10 constructs were identified as barriers, 8 as facilitators and 3 as both barriers and facilitators. Major barriers included unsatisfactory clinical performance (Innovation); lack of collaborative network between primary and tertiary hospitals, lack of information security measures and certification (outer setting); suboptimal data quality, misalignment between software functions and goals of healthcare institutions (inner setting); unmet clinical needs (individuals). Key facilitators were strong empirical evidence of effectiveness, improved clinical efficiency (innovation); national guidelines related to AI, deployment of AI software in peer hospitals (outer setting); integration of AI software into existing hospital systems (inner setting) and involvement of clinicians (implementation process). Conclusions The study findings contributed to the ongoing exploration of AI integration in healthcare from the perspective of China, emphasising the need for a comprehensive approach considering both innovation-specific factors and the broader organisational and contextual dynamics. As China and other developing countries continue to advance in adopting AI technologies, the derived insights could further inform healthcare practitioners, industry stakeholders and policy-makers, guiding policies and practices that promote the successful implementation of imaging-based diagnostic AI-assisted decision-making software in healthcare for optimal patient care.

medicine, general & internal
Implementing AI in healthcare-the relevance of trust: a scoping review

Emilie Steerling,Elin Siira,Per Nilsen,Petra Svedberg,Jens Nygren

DOI: https://doi.org/10.3389/frhs.2023.1211150

2023-08-24

Abstract:Background: The process of translation of AI and its potential benefits into practice in healthcare services has been slow in spite of its rapid development. Trust in AI in relation to implementation processes is an important aspect. Without a clear understanding, the development of effective implementation strategies will not be possible, nor will AI advance despite the significant investments and possibilities. Objective: This study aimed to explore the scientific literature regarding how trust in AI in relation to implementation in healthcare is conceptualized and what influences trust in AI in relation to implementation in healthcare. Methods: This scoping review included five scientific databases. These were searched to identify publications related to the study aims. Articles were included if they were published in English, after 2012, and peer-reviewed. Two independent reviewers conducted an abstract and full-text review, as well as carrying out a thematic analysis with an inductive approach to address the study aims. The review was reported in accordance with the PRISMA-ScR guidelines. Results: A total of eight studies were included in the final review. We found that trust was conceptualized in different ways. Most empirical studies had an individual perspective where trust was directed toward the technology's capability. Two studies focused on trust as relational between people in the context of the AI application rather than as having trust in the technology itself. Trust was also understood by its determinants and as having a mediating role, positioned between characteristics and AI use. The thematic analysis yielded three themes: individual characteristics, AI characteristics and contextual characteristics, which influence trust in AI in relation to implementation in healthcare. Conclusions: Findings showed that the conceptualization of trust in AI differed between the studies, as well as which determinants they accounted for as influencing trust. Few studies looked beyond individual characteristics and AI characteristics. Future empirical research addressing trust in AI in relation to implementation in healthcare should have a more holistic view of the concept to be able to manage the many challenges, uncertainties, and perceived risks.
Exploring Stakeholder Requirements to Enable Research and Development of Artificial Intelligence Algorithms in a Hospital-Based Generic Infrastructure: Results of a Multistep Mixed Methods Study

Lina Weinert,Maximilian Klass,Gerd Schneider,Oliver Heinze

DOI: https://doi.org/10.2196/43958

2023-04-18

Abstract:Background: Legal, controlled, and regulated access to high-quality data from academic hospitals currently poses a barrier to the development and testing of new artificial intelligence (AI) algorithms. To overcome this barrier, the German Federal Ministry of Health supports the "pAItient" (Protected Artificial Intelligence Innovation Environment for Patient Oriented Digital Health Solutions for developing, testing and evidence-based evaluation of clinical value) project, with the goal to establish an AI Innovation Environment at the Heidelberg University Hospital, Germany. It is designed as a proof-of-concept extension to the preexisting Medical Data Integration Center. Objective: The first part of the pAItient project aims to explore stakeholders' requirements for developing AI in partnership with an academic hospital and granting AI experts access to anonymized personal health data. Methods: We designed a multistep mixed methods approach. First, researchers and employees from stakeholder organizations were invited to participate in semistructured interviews. In the following step, questionnaires were developed based on the participants' answers and distributed among the stakeholders' organizations. In addition, patients and physicians were interviewed. Results: The identified requirements covered a wide range and were conflicting sometimes. Relevant patient requirements included adequate provision of necessary information for data use, clear medical objective of the research and development activities, trustworthiness of the organization collecting the patient data, and data should not be reidentifiable. Requirements of AI researchers and developers encompassed contact with clinical users, an acceptable user interface (UI) for shared data platforms, stable connection to the planned infrastructure, relevant use cases, and assistance in dealing with data privacy regulations. In a next step, a requirements model was developed, which depicts the identified requirements in different layers. This developed model will be used to communicate stakeholder requirements within the pAItient project consortium. Conclusions: The study led to the identification of necessary requirements for the development, testing, and validation of AI applications within a hospital-based generic infrastructure. A requirements model was developed, which will inform the next steps in the development of an AI innovation environment at our institution. Results from our study replicate previous findings from other contexts and will add to the emerging discussion on the use of routine medical data for the development of AI applications. International registered report identifier (irrid): RR2-10.2196/42208.
Translating ethical and quality principles for the effective, safe and fair development, deployment and use of artificial intelligence technologies in healthcare

Nicoleta J Economou-Zavlanos,Sophia Bessias,Michael P Cary,Armando D Bedoya,Benjamin A Goldstein,John E Jelovsek,Cara L O’Brien,Nancy Walden,Matthew Elmore,Amanda B Parrish,Scott Elengold,Kay S Lytle,Suresh Balu,Michael E Lipkin,Afreen Idris Shariff,Michael Gao,David Leverenz,Ricardo Henao,David Y Ming,David M Gallagher,Michael J Pencina,Eric G Poon

DOI: https://doi.org/10.1093/jamia/ocad221

2023-11-29

Journal of the American Medical Informatics Association

Abstract:Abstract Objective The complexity and rapid pace of development of algorithmic technologies pose challenges for their regulation and oversight in healthcare settings. We sought to improve our institution’s approach to evaluation and governance of algorithmic technologies used in clinical care and operations by creating an Implementation Guide that standardizes evaluation criteria so that local oversight is performed in an objective fashion. Materials and Methods Building on a framework that applies key ethical and quality principles (clinical value and safety, fairness and equity, usability and adoption, transparency and accountability, and regulatory compliance), we created concrete guidelines for evaluating algorithmic technologies at our institution. Results An Implementation Guide articulates evaluation criteria used during review of algorithmic technologies and details what evidence supports the implementation of ethical and quality principles for trustworthy health AI. Application of the processes described in the Implementation Guide can lead to algorithms that are safer as well as more effective, fair, and equitable upon implementation, as illustrated through 4 examples of technologies at different phases of the algorithmic lifecycle that underwent evaluation at our academic medical center. Discussion By providing clear descriptions/definitions of evaluation criteria and embedding them within standardized processes, we streamlined oversight processes and educated communities using and developing algorithmic technologies within our institution. Conclusions We developed a scalable, adaptable framework for translating principles into evaluation criteria and specific requirements that support trustworthy implementation of algorithmic technologies in patient care and healthcare operations.

information science & library science,computer science, information systems, interdisciplinary applications,health care sciences & services,medical informatics
Artificial Intelligence and the Implementation Challenge

James Shaw,Frank Rudzicz,Trevor Jamieson,Avi Goldfarb

DOI: https://doi.org/10.2196/13659

IF: 7.076

2019-07-10

Journal of Medical Internet Research

Abstract:BACKGROUND: Applications of artificial intelligence (AI) in health care have garnered much attention in recent years, but the implementation issues posed by AI have not been substantially addressed.OBJECTIVE: In this paper, we have focused on machine learning (ML) as a form of AI and have provided a framework for thinking about use cases of ML in health care. We have structured our discussion of challenges in the implementation of ML in comparison with other technologies using the framework of Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies (NASSS).METHODS: After providing an overview of AI technology, we describe use cases of ML as falling into the categories of decision support and automation. We suggest these use cases apply to clinical, operational, and epidemiological tasks and that the primary function of ML in health care in the near term will be decision support. We then outline unique implementation issues posed by ML initiatives in the categories addressed by the NASSS framework, specifically including meaningful decision support, explainability, privacy, consent, algorithmic bias, security, scalability, the role of corporations, and the changing nature of health care work.RESULTS: Ultimately, we suggest that the future of ML in health care remains positive but uncertain, as support from patients, the public, and a wide range of health care stakeholders is necessary to enable its meaningful implementation.CONCLUSIONS: If the implementation science community is to facilitate the adoption of ML in ways that stand to generate widespread benefits, the issues raised in this paper will require substantial attention in the coming years.

health care sciences & services,medical informatics
From Code to Bedside: Implementing Artificial Intelligence Using Quality Improvement Methods

Margaret Smith,Amelia Sattler,Grace Hong,Steven Lin

DOI: https://doi.org/10.1007/s11606-020-06394-w

IF: 5.7

2021-01-19

Journal of General Internal Medicine

Abstract:Despite increasing interest in how artificial intelligence (AI) can augment and improve healthcare delivery, the development of new AI models continues to outpace adoption in existing healthcare processes. Integration is difficult because current approaches separate the development of AI models from the complex healthcare environments in which they are intended to function, resulting in models developed without a clear and compelling use case and not tested or scalable in a clinical setting. We propose that current approaches and traditional research methods do not support successful AI implementation in healthcare and outline a repeatable mixed-methods approach, along with several examples, that facilitates uptake of AI technologies into human-driven healthcare processes. Unlike traditional research, these methods do not seek to control for variation, but rather understand it to learn how a technology will function in practice coupled with user-centered design techniques. This approach, leveraging design thinking and quality improvement methods, aims to increase the adoption of AI in healthcare and prompt further study to understand which methods are most successful for AI implementations.

medicine, general & internal,health care sciences & services
Stakeholders’ views on the organisational factors affecting application of artificial intelligence in healthcare: a scoping review protocol

Reda Lebcir,Tetiana Hill,Rifat Atun,Marija Cubric

DOI: https://doi.org/10.1136/bmjopen-2020-044074

IF: 3.006

2021-03-01

BMJ Open

Abstract:Introduction Artificial intelligence (AI) offers great potential for transforming healthcare delivery leading to better patient-outcomes and more efficient care delivery. However, despite these advantages, integration of AI in healthcare has not kept pace with technological advancements. Previous research indicates the importance of understanding various organisational factors that shape integration of new technologies in healthcare. Therefore, the aim of this study is to provide an overview of the existing organisational factors influencing adoption of AI in healthcare from the perspectives of different relevant stakeholders. By conducting this review, the various organisational factors that facilitate or hinder AI implementation in healthcare could be identified. Methods and analysis This study will follow the Joanna Briggs Institute framework, which includes the following stages: (1) defining and aligning objectives and questions, (2) developing and aligning the inclusions criteria with objectives and questions, (3) describing the planned approach to evidence searching and selection, (4) searching for the evidence, (5) selecting the evidence, (6) extracting the evidence, (7) charting the evidence, and summarising the evidence with regard to the objectives and questions. The databases searched will be MEDLINE (Ovid), CINAHL (Plus), PubMed, Cohrane Library, Scopus, MathSciNet, NICE Evidence, OpenGrey, O’REILLY and Social Care Online from January 2000 to June 2021. Search results will be reported based on The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews guidelines. The review will adopt diffusion of innovations theory, technology acceptance model and stakeholder theory as guiding conceptual models. Narrative synthesis will be used to integrate the findings. Ethics and dissemination Ethics approval will not be sought for this scoping review as it only includes information from previously published studies. The results will be disseminated through publication in a peer-reviewed journal. In addition, to ensure its findings reach relevant stakeholders, they will be presented at relevant conferences.

medicine, general & internal
Checklist to Support the Development and Implementation of AI in Clinical Settings

AYOMIDE OWOYEMI,Joanne Osuchukwu,Meghan Salwei,Andrew Boyd

DOI: https://doi.org/10.1101/2024.08.08.24311701

2024-08-09

Abstract:The integration of Artificial Intelligence (AI) in healthcare settings de-mands a nuanced approach that considers both technical performance and soci-otechnical factors. Recognizing this, our study introduces the Clinical AI Soci-otechnical Framework (CASoF), developed through literature synthesis, and re-fined via a Modified Delphi study involving global healthcare professionals. Our research identifies a critical gap in existing frameworks, which largely focus on either technical specifications or trial outcomes, neglecting the comprehensive sociotechnical dynamics essential for successful AI deployment in clinical envi-ronments. CASoF addresses this gap by providing a structured checklist that guides the planning, design, development, and implementation stages of AI sys-tems in healthcare. The checklist emphasizes the importance of considering the value proposition, data integrity, human-AI interaction, technical architecture, organizational culture, and ongoing support and monitoring, ensuring that AI tools are not only technologically sound but also practically viable and socially adaptable within clinical settings. Our findings suggest that the successful inte-gration of AI in healthcare depends on a balanced focus on both technological advancements and the socio-technical environment of clinical settings. CASoF represents a step forward in bridging this divide, offering a holistic approach to AI deployment that is mindful of the complexities of healthcare systems. The checklist aims to facilitate the development of AI tools that are effective, user-friendly, and seamlessly integrated into clinical workflows, ultimately enhancing patient care and healthcare outcomes.

Health Informatics
Guidelines for Artificial Intelligence in Medicine: Literature Review and Content Analysis of Frameworks

Norah L Crossnohere,Mohamed Elsaid,Jonathan Paskett,Seuli Bose-Brill,John F P Bridges

DOI: https://doi.org/10.2196/36823

IF: 7.076

2022-08-26

Journal of Medical Internet Research

Abstract:Background: Artificial intelligence (AI) is rapidly expanding in medicine despite a lack of consensus on its application and evaluation. Objective: We sought to identify current frameworks guiding the application and evaluation of AI for predictive analytics in medicine and to describe the content of these frameworks. We also assessed what stages along the AI translational spectrum (ie, AI development, reporting, evaluation, implementation, and surveillance) the content of each framework has been discussed. Methods: We performed a literature review of frameworks regarding the oversight of AI in medicine. The search included key topics such as "artificial intelligence," "machine learning," "guidance as topic," and "translational science," and spanned the time period 2014-2022. Documents were included if they provided generalizable guidance regarding the use or evaluation of AI in medicine. Included frameworks are summarized descriptively and were subjected to content analysis. A novel evaluation matrix was developed and applied to appraise the frameworks' coverage of content areas across translational stages. Results: Fourteen frameworks are featured in the review, including six frameworks that provide descriptive guidance and eight that provide reporting checklists for medical applications of AI. Content analysis revealed five considerations related to the oversight of AI in medicine across frameworks: transparency, reproducibility, ethics, effectiveness, and engagement. All frameworks include discussions regarding transparency, reproducibility, ethics, and effectiveness, while only half of the frameworks discuss engagement. The evaluation matrix revealed that frameworks were most likely to report AI considerations for the translational stage of development and were least likely to report considerations for the translational stage of surveillance. Conclusions: Existing frameworks for the application and evaluation of AI in medicine notably offer less input on the role of engagement in oversight and regarding the translational stage of surveillance. Identifying and optimizing strategies for engagement are essential to ensure that AI can meaningfully benefit patients and other end users.

health care sciences & services,medical informatics

Simplifying proximal isovelocity surface area as an assessment method of mitral valve area in patients with rheumatic mitral stenosis by fixing aliasing velocity and mitral valve angle.

Toward Successful Implementation of Artificial Intelligence in Health Care Practice: Protocol for a Research Program

A comprehensive overview of barriers and strategies for AI implementation in healthcare: Mixed-method design

Artificial Intelligence Applications in Health Care Practice: Scoping Review

From Bit To Bedside: A Practical Framework For Artificial Intelligence Product Development In Healthcare

ModelEd, TestEd, TrustEd

An ethics assessment tool for artificial intelligence implementation in healthcare: CARE-AI

Human Decision-making in an Artificial Intelligence-Driven Future in Health: Protocol for Comparative Analysis and Simulation

Creation of an Evidence-Based Implementation Framework for Digital Health Technology in the Intensive Care Unit: Qualitative Study

The Clinical Practice Integration of Artificial Intelligence (CPI-AI) framework

The 7-stranded structure of relaxed scallop muscle myosin filaments: support for a common head configuration in myosin-regulated muscles.

Evaluating Artificial Intelligence in Clinical Settings-Let Us Not Reinvent the Wheel

Barriers and facilitators to implementing imaging-based diagnostic artificial intelligence-assisted decision-making software in hospitals in China: a qualitative study using the updated Consolidated Framework for Implementation Research

Implementing AI in healthcare-the relevance of trust: a scoping review

Exploring Stakeholder Requirements to Enable Research and Development of Artificial Intelligence Algorithms in a Hospital-Based Generic Infrastructure: Results of a Multistep Mixed Methods Study

Translating ethical and quality principles for the effective, safe and fair development, deployment and use of artificial intelligence technologies in healthcare

Artificial Intelligence and the Implementation Challenge

From Code to Bedside: Implementing Artificial Intelligence Using Quality Improvement Methods

Stakeholders’ views on the organisational factors affecting application of artificial intelligence in healthcare: a scoping review protocol

Checklist to Support the Development and Implementation of AI in Clinical Settings

Guidelines for Artificial Intelligence in Medicine: Literature Review and Content Analysis of Frameworks