“Trust but Verify”: The Difficulty of Trusting Autonomous Weapons Systems

Heather M. Roff,David Danks

DOI: https://doi.org/10.1080/15027570.2018.1481907

2018-01-02

Journal of Military Ethics

Abstract:Autonomous weapons systems (AWSs) pose many challenges in complex battlefield environments. Previous discussions of them have largely focused on technological or policy issues. In contrast, we focus here on the challenge of trust in an AWS. One type of human trust depends only on judgments about the predictability or reliability of the trustee, and so are suitable for all manner of artifacts. However, AWSs that are worthy of the descriptor “autonomous” will not exhibit the required strong predictability in the complex, changing contexts of war. Instead, warfighters need to develop deeper, interpersonal trust that is grounded in understanding the values, beliefs, and dispositions of the AWS. Current acquisition, training, and deployment processes preclude the development of such trust, and so there are currently no routes for a warfighter to develop trust in an AWS. We thus survey three possible changes to current practices in order to facilitate the type of deep trust that is required for appropriate, ethical use of AWSs.