Abstract:Although AI algorithms and applications become more and popular in the healthcare sector, only few institutions have an operational AI strategy. Identifying the best suited processes for ML algorithm implementation and adoption is a big challenge. Also, raising human confidence in AI systems is elementary to building trustworthy, socially beneficial and responsible AI. A commonly agreed AI auditing framework that provides best practices and tools could help speeding up the adoption process. In this paper, we first highlight important concepts in the field of AI auditing and then restructure and subsume them into an ML auditing core criteria catalog. We conducted a scoping study where we analyzed sources being associated with the term "Auditable AI" in a qualitative way. We utilized best practices from Mayring (2000), Miles and Huberman (1994), and Bortz and Döring (2006). Based on referrals, additional relevant white papers and sources in the field of AI auditing were also included. The literature base was compared using inductively constructed categories. Afterwards, the findings were reflected on and synthesized into a resulting ML auditing core criteria catalog. The catalog is grouped into the categories: Conceptual Basics, Data & Algorithm Design and Assessment Metrics. As a practical guide, it consists of 30 questions developed to cover the mentioned categories and to guide ML implementation teams. Our consensus-based ML auditing criteria catalog is intended as a starting point for the development of evaluation strategies by specific stakeholders. We believe it will be beneficial to healthcare organizations that have been or will start implementing ML algorithms. Not only to help them being prepared for any upcoming legally required audit activities, but also to create better, well-perceived and accepted products. Potential limitations could be overcome by utilizing the proposed catalog in practice on real use cases to expose gaps and to further improve the catalog. Thus, this paper is seen as a starting point towards the development of a framework, where essential technical components can be specified.

Audit, Don't Explain -- Recommendations Based on a Socio-Technical Understanding of ML-Based Systems

Explainability Auditing for Intelligent Systems: A Rationale for Multi-Disciplinary Perspectives

Pragmatic auditing: a pilot-driven approach for auditing Machine Learning systems

Auditing Recommender Systems -- Putting the DSA into practice with a risk-scenario-based approach

XAudit : A Theoretical Look at Auditing with Explanations

Position Paper: Towards Transparent Machine Learning

AI, Explainability and Public Reason: The Argument from the Limitations of the Human Mind

Designing an ML Auditing Criteria Catalog as Starting Point for the Development of a Framework

Auditing the AI auditors: A framework for evaluating fairness and bias in high stakes AI predictive models.

Auditing large language models: a three-layered approach

Ethics-Based Auditing to Develop Trustworthy AI

Proposed Guidelines for the Responsible Use of Explainable Machine Learning

Towards algorithm auditing: managing legal, ethical and technological risks of AI, ML and associated algorithms

From Transparency to Accountability and Back: A Discussion of Access and Evidence in AI Auditing

Auditing of AI: Legal, Ethical and Technical Approaches

Auditing Local Explanations is Hard

Public vs Private Bodies: Who Should Run Advanced AI Evaluations and Audits? A Three-Step Logic Based on Case Studies of High-Risk Industries

How to explain AI systems to end users: a systematic literature review and research agenda

Insights into Data through Model Behaviour: An Explainability-driven Strategy for Data Auditing for Responsible Computer Vision Applications

The Case Against Explainability

The Right Tool for the Job: Open-Source Auditing Tools in Machine Learning