Subverting Skynet: The Strategic Promise of Lethal Autonomous Weapons and the Perils of Exploitation

Lennart Maschmeyer
DOI: https://doi.org/10.23919/CyCon55549.2022.9811008
2022-05-31
Abstract:Lethal Autonomous Weapons Systems (LAWS) promise a revolution in warfare by increasing the lethality of force while reducing the costs of war. Yet these gains come at the cost of significant yet underappreciated perils. LAWS are vulnerable to subversion, allowing adversaries to degrade or disable these systems or even turn them against their makers. Subversion involves exploiting flaws in complex systems to make them behave in unexpected ways. It is possible because, like other computer systems, the behavior of LAWS is determined by logical rules and routines. These rules and routines inevitably contain flaws, creating vulnerabilities that adversaries can exploit. This paper identifies three avenues of subversion: (1) manipulating the algorithm itself during the design process, (2) poisoning the data used to train the artificial intelligence operating the LAWS, and (3) manipulating physical objects LAWS are trained to respond to. This potential for subversion creates fundamental uncertainty for strategic planners, military commanders, and soldiers in the field. LAWS are powerful capabilities that may win wars, yet they may also become liabilities that lead to defeat against crafty adversaries. Hence, fulfilling the strategic promise of LAWS requires mitigating their vulnerabilities. Examining possible mitigations, the paper shows that technical fixes are currently unavailable, necessitating strategic solutions. It identifies two possible solutions. The first is employing counterintelligence strategies and tactics to detect, neutralize, and pre-empt attempts at subversion. The second is adopting a force structure model that maintains human superiority to neutralize rogue LAWS if necessary. However, both solutions reduce operational effectiveness and the strategic value of LAWS, thus forfeiting some of the core advantages these new systems promise. Consequently, the paper concludes that the strategic challenges of deploying LAWS currently outweigh the opportunities, necessitating a cautious approach and a greater prioritization of strategy development.
Political Science,Computer Science
What problem does this paper attempt to address?