Explainable Artificial Intelligence (XAI): An Engineering Perspective

F. Hussain,R. Hussain,E. Hossain

DOI: https://doi.org/10.48550/arXiv.2101.03613

2021-01-11

Abstract:The remarkable advancements in Deep Learning (DL) algorithms have fueled enthusiasm for using Artificial Intelligence (AI) technologies in almost every domain; however, the opaqueness of these algorithms put a question mark on their applications in safety-critical systems. In this regard, the `explainability' dimension is not only essential to both explain the inner workings of black-box algorithms, but it also adds accountability and transparency dimensions that are of prime importance for regulators, consumers, and service providers. eXplainable Artificial Intelligence (XAI) is the set of techniques and methods to convert the so-called black-box AI algorithms to white-box algorithms, where the results achieved by these algorithms and the variables, parameters, and steps taken by the algorithm to reach the obtained results, are transparent and explainable. To complement the existing literature on XAI, in this paper, we take an `engineering' approach to illustrate the concepts of XAI. We discuss the stakeholders in XAI and describe the mathematical contours of XAI from engineering perspective. Then we take the autonomous car as a use-case and discuss the applications of XAI for its different components such as object detection, perception, control, action decision, and so on. This work is an exploratory study to identify new avenues of research in the field of XAI.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

The main problem this paper attempts to address is the limitation of applying deep learning algorithms in safety-critical systems due to a lack of interpretability. Specifically, the paper focuses on how to transform so-called "black-box" artificial intelligence (AI) algorithms into "white-box" algorithms, making the results, variables, parameters, and the process by which the algorithm reaches its results transparent and interpretable. This involves increasing the interpretability, transparency, and accountability of the algorithms to meet the needs of regulators, consumers, and service providers.

### Main Issues
1. **Opacity of Deep Learning Algorithms**:
- Deep learning algorithms have made significant progress in many fields, but their internal workings' opacity raises concerns about their application in safety-critical systems.
- This opacity leads to the algorithms' lack of interpretability, making it difficult for users to understand why the algorithm makes specific decisions.

2. **Needs of Safety-Critical Systems**:
- In safety-critical systems such as autonomous vehicles and medical diagnostics, even minor algorithmic failures can lead to catastrophic consequences.
- Therefore, these systems require high levels of interpretability, transparency, and accountability to ensure the reliability and traceability of their decision-making processes.

3. **Legal and Ethical Requirements**:
- With the implementation of regulations such as the General Data Protection Regulation (GDPR), the interpretability and transparency of algorithms have become particularly important, especially in applications dealing with sensitive data.
- Legal and ethical requirements demand that algorithms not only perform well but also be able to explain their decision-making processes to ensure fairness and lack of bias.

### Solutions
The paper proposes an engineering approach to exploring explainable artificial intelligence (XAI), aiming to transform black-box AI algorithms into white-box algorithms. Specific methods include:

1. **Interpretability**:
- Making the internal workings of the algorithm clear through technical means, allowing users to understand why the algorithm makes specific decisions.

2. **Transparency**:
- Making the structure, variables, and parameters of the algorithm transparent so that users can verify the algorithm's correctness and reliability.

3. **Accountability**:
- Increasing the accountability of the algorithm to ensure that responsibility can be clearly assigned in case of issues and that the specific decision-making process can be traced.

### Application Case
The paper uses autonomous vehicles as an example to discuss in detail the application of XAI in different components (such as object detection, perception, control, decision-making, etc.). Through this case, the paper demonstrates how to implement XAI in practical applications to enhance system credibility and safety.

### Conclusion
The paper explores the importance of XAI from an engineering perspective and proposes specific solutions aimed at promoting further research and application of XAI in critical applications.