Clinicians' Voice: Fundamental Considerations for XAI in Healthcare

T. E. Röber,R. Goedhart,S. İ. Birbil
2024-11-08
Abstract:Explainable AI (XAI) holds the promise of advancing the implementation and adoption of AI-based tools in practice, especially in high-stakes environments like healthcare. However, most of the current research is disconnected from its practical applications and lacks input of end users. To address this, we conducted semi-structured interviews with clinicians to discuss their thoughts, hopes, and concerns. We find that clinicians generally think positively about developing AI-based tools for clinical practice, but they have concerns about how these will fit into their workflow and how it will impact clinician-patient relations. We further identify education of clinicians on AI as a crucial factor for the success of AI in healthcare and highlight aspects clinicians are looking for in (X)AI-based tools. In contrast to other studies, we take on a holistic and exploratory perspective to identify general requirements, which is necessary before moving on to testing specific (X)AI products for healthcare.
Machine Learning
What problem does this paper attempt to address?
### Problems the paper attempts to solve This paper aims to solve the key problems in the application of Explainable AI (XAI) in the healthcare field. Specifically, through semi - structured interviews with clinicians, the paper explores the following aspects: 1. **The current application status of XAI in healthcare**: - Understand the actual application of AI technology in healthcare, especially how AI tools are integrated into the clinical workflow. - Explore clinicians' attitudes, expectations, and concerns about AI tools. 2. **Clinicians' needs and expectations**: - Determine what features clinicians expect XAI tools to have, such as tasks like predicting complications and monitoring patients' health. - Understand clinicians' views and preferences on different explanation methods (such as feature importance scoring, counterfactual explanations, rule - based methods). 3. **Education and training needs**: - Emphasize the importance of AI - related training for clinicians to ensure that they can correctly understand and use AI tools. - Solve the problems of misuse or over - reliance on AI tools due to lack of AI knowledge. 4. **Trust and responsibility issues**: - Explore clinicians' trust in AI tools, especially how to balance trust and responsibility in high - risk decision - making. - Analyze the impact of the verification and certification process of AI tools on building trust. 5. **The importance of multidisciplinary cooperation**: - Emphasize the crucial role of interdisciplinary cooperation in the development and implementation of AI tools to ensure that these tools can be seamlessly integrated into clinical practice. ### Conclusion Through in - depth interviews, the paper identifies the key requirements for the successful implementation of XAI tools and provides specific suggestions for future research and development. The main conclusions include: - **Administrative tasks and risk warning**: Clinicians hope that AI tools can reduce administrative burdens and be used in risk assessment and warning systems. - **Visual explanation**: Visual explanation methods are easier to understand and help eliminate language barriers. - **Multidisciplinary cooperation**: Interdisciplinary cooperation is crucial for the development of successful AI tools. - **AI training**: AI training for clinicians is the key to ensuring their correct use of AI tools. - **Trust and responsibility**: Building trust in AI tools requires a transparent verification and certification process, while clearly defining the attribution of responsibilities. Through these research results, the paper provides valuable guidance for the further development and application of XAI in the healthcare field.