Abstract:In recent years, penetration testing (pen-testing) has emerged as a crucial process for evaluating the security level of network infrastructures by simulating real-world cyber-attacks. Automating pen-testing through reinforcement learning (RL) facilitates more frequent assessments, minimizes human effort, and enhances scalability. However, real-world pen-testing tasks often involve incomplete knowledge of the target network system. Effectively managing the intrinsic uncertainties via partially observable Markov decision processes (POMDPs) constitutes a persistent challenge within the realm of pen-testing. Furthermore, RL agents are compelled to formulate intricate strategies to contend with the challenges posed by partially observable environments, thereby engendering augmented computational and temporal expenditures. To address these issues, this study introduces EPPTA (efficient POMDP-driven penetration testing agent), an agent built on an asynchronous RL framework, designed for conducting pen-testing tasks within partially observable environments. We incorporate an implicit belief module in EPPTA, grounded on the belief update formula of the traditional POMDP model, which represents the agent's probabilistic estimation of the current environment state. Furthermore, by integrating the algorithm with the high-performance RL framework, sample factory, EPPTA significantly reduces convergence time compared to existing pen-testing methods, resulting in an approximately 20-fold acceleration. Empirical results across various pen-testing scenarios validate EPPTA's superior task reward performance and enhanced scalability, providing substantial support for efficient and advanced evaluation of network infrastructure security. The article introduces EPPTA, a reinforcement learning framework designed for penetration testing, and assesses its performance across diverse network configurations. EPPTA incorporates a belief module, augmenting its capacity to handle partially observable security challenges. EPPTA outperforms other methods in terms of convergence times, especially in larger and more complex scenarios, showcasing its scalability and adaptability to evolving security challenges.image

EPPTA: Efficient Partially Observable Reinforcement Learning Agent for Penetration Testing Applications

Ontology and Reinforcement Learning Based Intelligent Agent Automatic Penetration Test

Knowledge-Informed Auto-Penetration Testing Based on Reinforcement Learning with Reward Machine

DynPen: Automated Penetration Testing in Dynamic Network Scenarios Using Deep Reinforcement Learning

Hierarchical reinforcement learning for efficient and effective automated penetration testing of large networks

A Hierarchical Deep Reinforcement Learning Model with Expert Prior Knowledge for Intelligent Penetration Testing

Behaviour-diverse automatic penetration testing: a coverage-based deep reinforcement learning approach

An Automated Penetration Testing Framework Based on Hierarchical Reinforcement Learning

Towards Generalizable Autonomous Penetration Testing via Domain Randomization and Meta-Reinforcement Learning

Raijū: Reinforcement Learning-Guided Post-Exploitation for Automating Security Assessment of Network Systems

AutoPT: How Far Are We from the End2End Automated Web Penetration Testing?

GAIL-PT: An Intelligent Penetration Testing Framework with Generative Adversarial Imitation Learning

Autonomous Penetration Testing Based on Improved Deep Q-Network

SetTron: Towards Better Generalisation in Penetration Testing with Reinforcement Learning

PentestAgent: Incorporating LLM Agents to Automated Penetration Testing

INNES: An intelligent network penetration testing model based on deep reinforcement learning

POMDPs Make Better Hackers: Accounting for Uncertainty in Penetration Testing

Applying Action Masking and Curriculum Learning Techniques to Improve Data Efficiency and Overall Performance in Operational Technology Cyber Security using Reinforcement Learning

Evaluation of Reinforcement Learning for Autonomous Penetration Testing using A3C, Q-learning and DQN

An Intelligent Penetration Testing Method Using Human Feedback

Cooperative multi-agent target searching: a deep reinforcement learning approach based on parallel hindsight experience replay