Abstract:Artificial intelligence (AI) has become nearly ubiquitous in modern society, from components of mobile applications to medical support systems, and everything in between. In societally impactful systems imbued with AI, there has been increasing concern related to opaque AI, that is, artificial intelligence where it is unclear how or why certain decisions are reached. This has led to a recent boom in research on "explainable AI" (XAI), or approaches to making AI more explainable and understandable to human users. In the military domain, numerous bodies have argued that autonomous and AI-enabled weapon systems ought not incorporate unexplainable AI, with the International Committee of the Red Cross and the United States Department of Defense both explicitly including explainability as a relevant factor in the development and use of such systems. In this article, I present a cautiously critical assessment of this view, arguing that explainability will be irrelevant for many current and near-future autonomous systems in the military (which do not incorporate any AI), that it will be trivially incorporated into most military systems which do possess AI (as these generally possess simpler AI systems), and that for those systems with genuinely opaque AI, explainability will prove to be of more limited value than one might imagine. In particular, I argue that explainability, while indeed a virtue in design, is a virtue aimed primarily at designers and troubleshooters of AI-enabled systems, but is far less relevant for users and handlers actually deploying these systems. I further argue that human–machine teaming is a far more important element of responsibly using AI for military purposes, adding that explainability may undermine efforts to improve human–machine teamings by creating a prima facie sense that the AI, due to its explainability, may be utilized with little (or less) potential for mistakes. I conclude by clarifying that the arguments are not against XAI in the military, but are instead intended as a caution against over-inflating the value of XAI in this domain, or ignoring the limitations and potential pitfalls of this approach.

On Explainable AI Solutions for Targeting in Cyber Military Operations

Explainable AI in the military domain

Explainable AI for Intelligence Augmentation in Multi-Domain Operations

Explainable artificial intelligence envisioned security mechanism for cyber threat hunting

More Questions than Answers? Lessons from Integrating Explainable AI into a Cyber-AI Tool

On the Importance of Domain-specific Explanations in AI-based Cybersecurity Systems (Technical Report)

Towards Global Explainability of Artificial Intelligence Agent Tactics in Close Air Combat

Explainable AI for Operational Research: A Defining Framework, Methods, Applications, and a Research Agenda

Explainable AI for cybersecurity automation, intelligence and trustworthiness in digital twin: Methods, taxonomy, challenges and prospects

Explainable Artificial Intelligence and Cybersecurity: A Systematic Literature Review

SoK: Explainable Machine Learning for Computer Security Applications

Explainable Intelligence-Driven Defense Mechanism against Advanced Persistent Threats: A Joint Edge Game and AI Approach

Cybertrust: From Explainable to Actionable and Interpretable AI (AI2)

On Two XAI Cultures: A Case Study of Non-technical Explanations in Deployed AI System

Responsible use of AI in military systems: prospects and challenges

Integrating Artificial Intelligence into Weapon Systems

Operationalizing Human-Centered Perspectives in Explainable AI

Explainable AI for Medical Image Analysis in Medical Cyber-Physical Systems: Enhancing Transparency and Trustworthiness of IoMT

Explainable Artificial Intelligence Applications in Cyber Security: State-of-the-Art in Research

Artificial Intelligence in Cybersecurity: The Use of AI Along the Cyber Kill Chain

Explainable artificial intelligence for education and training