Abstract:Machine learning has made tremendous progress in predictive performance in recent years. Despite these advances, employing machine learning models in high-stake domains remains challenging due to the opaqueness of many high-performance models. If their behavior cannot be analyzed, this likely decreases the trust in such models and hinders the acceptance of human decision-makers. Motivated by these challenges, we propose a process model for developing and evaluating explainable decision support systems that are tailored to the needs of different stakeholders. To demonstrate its usefulness, we apply the process model to a real-world application in an enterprise context. The goal is to increase the acceptance of an existing black-box model developed at a car manufacturer for supporting manual goodwill assessments. Following the proposed process, we conduct two quantitative surveys targeted at the application's stakeholders. Our study reveals that textual explanations based on local feature importance best fit the needs of the stakeholders in the considered use case. Specifically, our results show that all stakeholders, including business specialists, goodwill assessors, and technical IT experts, agree that such explanations significantly increase their trust in the decision support system. Furthermore, our technical evaluation confirms the faithfulness and stability of the selected explanation method. These practical findings demonstrate the potential of our process model to facilitate the successful deployment of machine learning models in enterprise settings. The results emphasize the importance of developing explanations that are tailored to the specific needs and expectations of diverse stakeholders.

An Explainable Decision Support System for Predictive Process Analytics

An explainable decision support system for predictive process analytics

Explainable Predictive Process Monitoring

Explainable Predictive Decision Mining for Operational Support

Evaluating Explainable Methods for Predictive Process Analytics: A Functionally-Grounded Approach

Local Post-Hoc Explanations for Predictive Process Monitoring in Manufacturing

Exploring Interpretability for Predictive Process Analytics

Stakeholder-centric explanations for black-box decisions: an XAI process model and its application to automotive goodwill assessments

Intelligent Decision Support for Energy Management: A Methodology for Tailored Explainability of Artificial Intelligence Analytics

Designing Explainable Predictive Machine Learning Artifacts: Methodology and Practical Demonstration

Explainable AI Enabled Inspection of Business Process Prediction Models

Generating Feasible and Plausible Counterfactual Explanations for Outcome Prediction of Business Processes

Explainability in Process Outcome Prediction: Guidelines to Obtain Interpretable and Faithful Models

DiCE4EL: Interpreting Process Predictions using a Milestone-Aware Counterfactual Approach

Building Interpretable Models for Business Process Prediction using Shared and Specialised Attention Mechanisms

Data Mining and Analytics in the Process Industry: the Role of Machine Learning

Interpretable and Explainable Machine Learning Methods for Predictive Process Monitoring: A Systematic Literature Review

AI Trust in Business Processes: The Need for Process-Aware Explanations

Explainable Artificial Intelligence for Improved Modeling of Processes

Robustness of Explainable Artificial Intelligence in Industrial Process Modelling

Interpretable Narrative Explanation for ML Predictors with LP: A Case Study for XAI