Abstract:Machine learning-based systems have presented increasing learning performance, in a wide variety of tasks. However, the problem with some state-of-the-art models is their lack of transparency, trustworthiness, and explainability. To address this problem, eXplainable Artificial Intelligence (XAI) appeared. It is a research field that aims to make black-box models more understandable to humans. The research on this topic has increased in recent years, and many methods, such as LIME (Local Interpretable Model-Agnostic Explanations) and SHAP (SHapley Additive exPlanations) have been proposed. Machine learning-based Intrusion Detection Systems (IDS) are one of the many application domains of XAI. However, most of the works about model interpretation focus on other fields, like computer vision, natural language processing, biology, healthcare, etc. This poses a challenge for cybersecurity professionals tasked with analyzing IDS results, thereby impeding their capacity to make informed decisions. In an attempt to address this problem, we have selected two XAI methods, LIME, and SHAP. Using the methods, we have retrieved explanations for the results of a black-box model, part of an IDS solution that performs intrusion detection on IoT devices, increasing its interpretability. In order to validate the explanations, we carried out a perturbation analysis where we tried to obtain a different classification based on the features present in the explanations. With the explanations and the perturbation analysis we were able to draw conclusions about the negative impact of particular features on the model results when present in the input data, making it easier for cybersecurity experts when analyzing the model results and it serves as an aid to the continuous improvement the model. The perturbations also serve as a comparison of performance between LIME and SHAP. To evaluate the degree of interpretability increase, and the explanations provided by each XAI method of the model and directly compare the XAI methods, we have performed a survey analysis.

Analyzing and assessing explainable AI models for smart agriculture environments

Enhancing crop recommendation systems with explainable artificial intelligence: a study on agricultural decision-making

Enhanced Infield Agriculture with Interpretable Machine Learning Approaches for Crop Classification

AgroXAI: Explainable AI-Driven Crop Recommendation System for Agriculture 4.0

Explainable Artificial Intelligence Model for Predictive Maintenance in Smart Agricultural Facilities

Explainable AI in Grassland Monitoring: Enhancing Model Performance and Domain Adaptability

EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications

OAK4XAI: Model towards Out-Of-Box eXplainable Artificial Intelligence for Digital Agriculture

User-centric evaluation of explainability of AI with and for humans: a comprehensive empirical study

Explainable AI: Machine Learning Interpretation in Blackcurrant Powders

Analyses of Diverse Agricultural Worker Data with Explainable Artificial Intelligence: XAI based on SHAP, LIME, and LightGBM

A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME

Explainability and transparency in the realm of digital humanities: toward a historian XAI

Evaluating explainable artificial intelligence methods for multi-label deep learning classification tasks in remote sensing

Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspective

Quantitative Analysis of Primary Attribution Explainable Artificial Intelligence Methods for Remote Sensing Image Classification

Explainable AI for Intrusion Detection Systems: LIME and SHAP Applicability on Multi-Layer Perceptron

More Questions than Answers? Lessons from Integrating Explainable AI into a Cyber-AI Tool

AI and Machine Learning for Optimal Crop Yield Optimization in the USA

Evaluating the Correctness of Explainable AI Algorithms for Classification

Explainable AI (XAI): History, Basic Ideas and Methods