Algor-ethics: charting the ethical path for AI in critical care

Jonathan Montomoli,Maria Maddalena Bitondo,Marco Cascella,Emanuele Rezoagli,Luca Romeo,Valentina Bellini,Federico Semeraro,Emiliano Gamberini,Emanuele Frontoni,Vanni Agnoletti,Mattia Altini,Paolo Benanti,Elena Giovanna Bignami

DOI: https://doi.org/10.1007/s10877-024-01157-y

IF: 1.977

2024-04-06

Journal of Clinical Monitoring and Computing

Abstract:The integration of Clinical Decision Support Systems (CDSS) based on artificial intelligence (AI) in healthcare is groundbreaking evolution with enormous potential, but its development and ethical implementation, presents unique challenges, particularly in critical care, where physicians often deal with life-threating conditions requiring rapid actions and patients unable to participate in the decisional process. Moreover, development of AI-based CDSS is complex and should address different sources of bias, including data acquisition, health disparities, domain shifts during clinical use, and cognitive biases in decision-making. In this scenario algor-ethics is mandatory and emphasizes the integration of 'Human-in-the-Loop' and 'Algorithmic Stewardship' principles, and the benefits of advanced data engineering. The establishment of Clinical AI Departments (CAID) is necessary to lead AI innovation in healthcare, ensuring ethical integrity and human-centered development in this rapidly evolving field.

anesthesiology

What problem does this paper attempt to address?

The paper primarily explores the application of Artificial Intelligence (AI) in the field of intensive care and the ethical and technical challenges it faces. The authors point out that AI technology has great potential in intensive care scenarios, capable of assisting doctors in making diagnostic and treatment decisions through Clinical Decision Support Systems (CDSS), thereby improving patient outcomes. However, there are many complexities and challenges in the actual deployment of AI systems:

1. **Unique Challenges**: In the intensive care environment, doctors often need to respond quickly to life-threatening situations, and patients may not be able to participate in the decision-making process. Therefore, ensuring the fairness and accountability of AI systems becomes particularly important.

2. **Bias in Data Acquisition**: The data collection practices of different medical institutions vary significantly, which can affect the performance of AI algorithms in different environments. To address this issue, the paper suggests using advanced data engineering methods, such as Extract, Transform, Load (ETL) pipelines, to effectively integrate data from different sources.

3. **Sources of Bias in Algorithm Development**: The paper discusses the issue of health inequalities potentially being exacerbated by AI algorithms due to insufficient consideration. To avoid this, the paper proposes the concept of "algorithmic fairness audits" and emphasizes the importance of evaluating model performance across different population subgroups.

4. **Cognitive Bias in Using AI**: Humans often rely on intuition rather than optimal strategies in decision-making. This reliance can lead to intuition-based decisions, introducing bias, especially in the field of intensive care. Additionally, over-reliance on AI may cause doctors to overlook the limitations of AI.

5. **Ensuring Ethical Integrity**: The paper introduces the new concept of "algor-ethics," aiming to establish an ethical framework for the application of AI in healthcare. It emphasizes the principles of "Human-in-the-Loop" (HITL) and "Algorithmic Stewardship" to ensure that AI development aligns with human values and social norms.

6. **Role of Clinical AI Departments**: The paper suggests the establishment of Clinical AI Departments (CAID) as institutions leading AI innovation, ensuring that the ethical integrity and development direction of AI technology in healthcare align with human-centered principles.

In summary, the paper highlights the challenges faced in applying AI technology in the field of intensive care and proposes a series of solutions to promote the safe and responsible development and implementation of AI technology.