An Explanatory Model Steering System for Collaboration between Domain Experts and AI

Aditya Bhattacharya,Simone Stumpf,Katrien Verbert
DOI: https://doi.org/10.1145/3631700.3664886
2024-05-17
Abstract:With the increasing adoption of Artificial Intelligence (AI) systems in high-stake domains, such as healthcare, effective collaboration between domain experts and AI is imperative. To facilitate effective collaboration between domain experts and AI systems, we introduce an Explanatory Model Steering system that allows domain experts to steer prediction models using their domain knowledge. The system includes an explanation dashboard that combines different types of data-centric and model-centric explanations and allows prediction models to be steered through manual and automated data configuration approaches. It allows domain experts to apply their prior knowledge for configuring the underlying training data and refining prediction models. Additionally, our model steering system has been evaluated for a healthcare-focused scenario with 174 healthcare experts through three extensive user studies. Our findings highlight the importance of involving domain experts during model steering, ultimately leading to improved human-AI collaboration.
Human-Computer Interaction,Artificial Intelligence
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to improve the quality and reliability of prediction models through effective collaboration between domain experts and artificial intelligence (AI) systems in high - risk fields (such as healthcare). Specifically, the paper proposes an Explanatory Model Steering system, which aims to enable domain experts to use their professional knowledge to adjust prediction models, thereby optimizing these models. This system mainly includes the following four aspects: 1. **Explanation Dashboard**: Provide the interpretability of prediction models to help users understand how the models work. 2. **Domain - expert - driven Model Steering**: Allow domain experts to adjust models according to their own knowledge. 3. **Manual and Automatic Data Configuration Methods**: Used for fine - tuning training data to improve the performance of models. 4. **Update Prediction Models and Explanations Based on Configuration Data**: Ensure that models and explanations are always up - to - date. Through the combination of these four aspects, this system aims to enhance the interaction between domain experts and AI systems, improve the transparency and interpretability of models, and thus achieve better results in practical applications. The paper also evaluates the effectiveness and practicality of this system in the healthcare field through three user studies.