Abstract:Cyber-attacks pose a security threat to military command and control networks, Intelligence, Surveillance, and Reconnaissance (ISR) systems, and civilian critical national infrastructure. The use of artificial intelligence and autonomous agents in these attacks increases the scale, range, and complexity of this threat and the subsequent disruption they cause. Autonomous Cyber Defence (ACD) agents aim to mitigate this threat by responding at machine speed and at the scale required to address the problem. Sequential decision-making algorithms such as Deep Reinforcement Learning (RL) provide a promising route to create ACD agents. These algorithms focus on a single objective such as minimizing the intrusion of red agents on the network, by using a handcrafted weighted sum of rewards. This approach removes the ability to adapt the model during inference, and fails to address the many competing objectives present when operating and protecting these networks. Conflicting objectives, such as restoring a machine from a back-up image, must be carefully balanced with the cost of associated down-time, or the disruption to network traffic or services that might result. Instead of pursing a Single-Objective RL (SORL) approach, here we present a simple example of a multi-objective network defence game that requires consideration of both defending the network against red-agents and maintaining critical functionality of green-agents. Two Multi-Objective Reinforcement Learning (MORL) algorithms, namely Multi-Objective Proximal Policy Optimization (MOPPO), and Pareto-Conditioned Networks (PCN), are used to create two trained ACD agents whose performance is compared on our Multi-Objective Cyber Defence game. The benefits and limitations of MORL ACD agents in comparison to SORL ACD agents are discussed based on the investigations of this game.

On Autonomous Agents in a Cyber Defence Environment

Towards Autonomous Cyber Operation Agents: Exploring the Red Case.

Multi-Objective Reinforcement Learning for Automated Resilient Cyber Defence

Autonomous Network Defence using Reinforcement Learning

Learning Cyber Defence Tactics from Scratch with Multi-Agent Reinforcement Learning

Multi-Agent Actor-Critics in Autonomous Cyber Defense

Deep Reinforcement Learning for Autonomous Cyber Defence: A Survey

Entity-based Reinforcement Learning for Autonomous Cyber Defence

A Multiagent CyberBattleSim for RL Cyber Operation Agents

Causally aware reinforcement learning agents for autonomous cyber defence

Training Automated Defense Strategies Using Graph-based Cyber Attack Simulations

Hierarchical Multi-agent Reinforcement Learning for Cyber Network Defense

Autonomous Cyber Defense Introduces Risk: Can We Manage the Risk?

The Path To Autonomous Cyber Defense

Doers, not Watchers: Intelligent Autonomous Agents are a Path to Cyber Resilience

Exploring reinforcement learning for incident response in autonomous military vehicles

Inroads into Autonomous Network Defence using Explained Reinforcement Learning

Automated Cyber Defence: A Review

Autonomous Attack Mitigation for Industrial Control Systems

Towards Type Agnostic Cyber Defense Agents

Adversarial Deep Reinforcement Learning for Cyber Security in Software Defined Networks