Abstract:Machine learning-based systems have presented increasing learning performance, in a wide variety of tasks. However, the problem with some state-of-the-art models is their lack of transparency, trustworthiness, and explainability. To address this problem, eXplainable Artificial Intelligence (XAI) appeared. It is a research field that aims to make black-box models more understandable to humans. The research on this topic has increased in recent years, and many methods, such as LIME (Local Interpretable Model-Agnostic Explanations) and SHAP (SHapley Additive exPlanations) have been proposed. Machine learning-based Intrusion Detection Systems (IDS) are one of the many application domains of XAI. However, most of the works about model interpretation focus on other fields, like computer vision, natural language processing, biology, healthcare, etc. This poses a challenge for cybersecurity professionals tasked with analyzing IDS results, thereby impeding their capacity to make informed decisions. In an attempt to address this problem, we have selected two XAI methods, LIME, and SHAP. Using the methods, we have retrieved explanations for the results of a black-box model, part of an IDS solution that performs intrusion detection on IoT devices, increasing its interpretability. In order to validate the explanations, we carried out a perturbation analysis where we tried to obtain a different classification based on the features present in the explanations. With the explanations and the perturbation analysis we were able to draw conclusions about the negative impact of particular features on the model results when present in the input data, making it easier for cybersecurity experts when analyzing the model results and it serves as an aid to the continuous improvement the model. The perturbations also serve as a comparison of performance between LIME and SHAP. To evaluate the degree of interpretability increase, and the explanations provided by each XAI method of the model and directly compare the XAI methods, we have performed a survey analysis.

Extending LIME for Business Process Automation

AI Trust in Business Processes: The Need for Process-Aware Explanations

Leveraging Large Language Models for Enhanced Process Model Comprehension

Evaluating Explainable Methods for Predictive Process Analytics: A Functionally-Grounded Approach

BMB-LIME: LIME with modeling local nonlinearity and uncertainty in explainability

An Extension of LIME with Improvement of Interpretability and Fidelity

Explainable AI Enabled Inspection of Business Process Prediction Models

Evaluating Large Language Models on Business Process Modeling: Framework, Benchmark, and Self-Improvement Analysis

Conversing with business process-aware large language models: the BPLLM framework

EvaluateXAI: A Framework to Evaluate the Reliability and Consistency of Rule-based XAI Techniques for Software Analytics Tasks

Process Modeling With Large Language Models

Are Your Explanations Reliable? Investigating the Stability of LIME in Explaining Text Classifiers by Marrying XAI and Adversarial Attack

Explainable AI for Intrusion Detection Systems: LIME and SHAP Applicability on Multi-Layer Perceptron

Explainable Artificial Intelligence for Improved Modeling of Processes

LIMEtree: Consistent and Faithful Multi-class Explanations

Local Interpretable Model Agnostic Shap Explanations for machine learning models

From Philosophy to Interfaces: an Explanatory Method and a Tool Inspired by Achinstein's Theory of Explanation

A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME

On the Applicability of Explainable Artificial Intelligence for Software Requirement Analysis

Explainable AI in Credit Risk Management

A Review of AI and Machine Learning Contribution in Predictive Business Process Management (Process Enhancement and Process Improvement Approaches)