Commercial AI, Conflict, and Moral Responsibility: A theoretical analysis and practical approach to the moral responsibilities associated with dual-use AI technology

Daniel Trusilo,David Danks
2024-01-31
Abstract:This paper presents a theoretical analysis and practical approach to the moral responsibilities when developing AI systems for non-military applications that may nonetheless be used for conflict applications. We argue that AI represents a form of crossover technology that is different from previous historical examples of dual- or multi-use technology as it has a multiplicative effect across other technologies. As a result, existing analyses of ethical responsibilities around dual-use technologies do not necessarily work for AI systems. We instead argue that stakeholders involved in the AI system lifecycle are morally responsible for uses of their systems that are reasonably foreseeable. The core idea is that an agent's moral responsibility for some action is not necessarily determined by their intentions alone; we must also consider what the agent could reasonably have foreseen to be potential outcomes of their action, such as the potential use of a system in conflict even when it is not designed for that. In particular, we contend that it is reasonably foreseeable that: (1) civilian AI systems will be applied to active conflict, including conflict support activities, (2) the use of civilian AI systems in conflict will impact applications of the law of armed conflict, and (3) crossover AI technology will be applied to conflicts that fall short of armed conflict. Given these reasonably foreseeably outcomes, we present three technically feasible actions that developers of civilian AIs can take to potentially mitigate their moral responsibility: (a) establishing systematic approaches to multi-perspective capability testing, (b) integrating digital watermarking in model weight matrices, and (c) utilizing monitoring and reporting mechanisms for conflict-related AI applications.
Computers and Society,Artificial Intelligence
What problem does this paper attempt to address?
The problem this paper attempts to address is: how to understand and respond to the moral responsibility that arises from the potential use of AI systems, developed for non-military applications, in conflict scenarios. Specifically, the paper points out: 1. **The uniqueness of AI as a cross-cutting technology**: Unlike previous dual-use or multi-use technologies, AI has cross-domain enhancement effects and can be widely applied in various technologies and scenarios. Therefore, existing ethical analyses of dual-use technologies do not fully apply to AI systems. 2. **Reasonably foreseeable outcomes**: The paper argues that developers, deployers, and other stakeholders of AI systems should bear moral responsibility for those uses of the systems that can be reasonably foreseen. Even if these systems are not designed for conflict purposes, the fact that they may be used in conflict scenarios is foreseeable. For example: - Civilian AI systems may be used in actual conflicts, including supporting conflict activities. - The use of these systems may affect the application of law in armed conflicts. - They may also be used in gray zone conflicts, which are conflicts between peace and war. 3. **Specific moral and ethical responsibilities**: The paper proposes three technically feasible actions to mitigate the moral responsibility of stakeholders: - Establishing a systematic approach to multi-perspective capability testing. - Integrating digital watermarks into model weight matrices. - Creating and utilizing monitoring and reporting mechanisms for conflict-related AI applications. Through these measures, the paper aims to provide a theoretical and practical framework for AI system developers to better consider and address potential moral risks during the development process.