It is not "accuracy vs. explainability" -- we need both for trustworthy AI systems

D. Petkovic

DOI: https://doi.org/10.48550/arXiv.2212.11136

2022-12-24

Abstract:We are witnessing the emergence of an AI economy and society where AI technologies are increasingly impacting health care, business, transportation and many aspects of everyday life. Many successes have been reported where AI systems even surpassed the accuracy of human experts. However, AI systems may produce errors, can exhibit bias, may be sensitive to noise in the data, and often lack technical and judicial transparency resulting in reduction in trust and challenges in their adoption. These recent shortcomings and concerns have been documented in scientific but also in general press such as accidents with self driving cars, biases in healthcare, hiring and face recognition systems for people of color, seemingly correct medical decisions later found to be made due to wrong reasons etc. This resulted in emergence of many government and regulatory initiatives requiring trustworthy and ethical AI to provide accuracy and robustness, some form of explainability, human control and oversight, elimination of bias, judicial transparency and safety. The challenges in delivery of trustworthy AI systems motivated intense research on explainable AI systems (XAI). Aim of XAI is to provide human understandable information of how AI systems make their decisions. In this paper we first briefly summarize current XAI work and then challenge the recent arguments of accuracy vs. explainability for being mutually exclusive and being focused only on deep learning. We then present our recommendations for the use of XAI in full lifecycle of high stakes trustworthy AI systems delivery, e.g. development, validation and certification, and trustworthy production and maintenance.

Machine Learning,Artificial Intelligence,Computers and Society

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the contradiction between accuracy and interpretability in current artificial intelligence systems. With the wide application of AI technology in many fields such as medical care, commerce, and transportation, the accuracy of AI systems has been significantly improved, even exceeding the performance of human experts. However, these systems also expose some problems, such as the possibility of making mistakes, the existence of biases, being sensitive to noise in data, and lacking technical and judicial transparency, etc. These problems have led to a decrease in the public's trust in AI systems and affected the popularization of their applications.

The paper points out that in recent years, governments and regulatory agencies have begun to require AI systems to not only provide accuracy and robustness, but also have a certain degree of interpretability, human control and supervision, elimination of biases, judicial transparency, and security. These requirements have promoted a research boom in Explainable AI (XAI). The goal of XAI is to provide information that humans can understand and explain how AI systems make decisions. This is a challenging task, especially in deep - learning systems, because the internal mechanisms of these systems are complex and difficult to observe.

The paper challenges the view that "accuracy and interpretability are mutually exclusive", believing that this opposition is limited to the field of deep learning, and in fact the two can coexist. The author puts forward a series of suggestions, aiming to integrate XAI analysis into the entire life cycle of high - risk and trustworthy AI systems, including the development, verification/certification, and production maintenance stages. In this way, not only can the interpretability of AI systems be improved, but also their reliability and security can be enhanced, thereby promoting the public's trust and support for AI systems.