TSGS: Two-stage security game solution based on deep reinforcement learning for Internet of Things
Xuecai Feng,Hui Xia,Shuo Xu,Lijuan Xu,Rui Zhang
DOI: https://doi.org/10.1016/j.eswa.2023.120965
IF: 8.5
2023-07-30
Expert Systems with Applications
Abstract:The lack of effective defense resource allocation strategies and reliable multi-agent collaboration mechanisms lead to the low stability of Deep Reinforcement Learning (DRL)-based security defense strategies in several Internet of Things (IoT) applications. To address the aforementioned issues and approach real-world scenarios, we construct a grid-based adversarial security scenario, propose a two-stage zero-sum security game model (including the resource allocation stage and the patrolling detection stage), and design a t wo-stage s ecurity g ame s olution algorithm based on DRL for this game model, named TSGS. In the resource allocation stage, TSGS uses auxiliary action embedding and gradient approximation approach to compute the Nash Equilibrium (NE) allocation strategy, which addresses the problem of unreasonable allocation. In the patrolling detection stage, TSGS achieves team collaboration by training a multi-agent Dueling Deep-Q Network under the centralized training and decentralized execution (CTDE) framework, which solves the cooperation problem among multiple defense agents. In addition, we design and implement a multi-parameterized attacker model to make the attacker's behaviors more realistic in the game. Finally, the validity of the TSGS is verified by detailed experimental results for several adversarial experimental scenarios. Compared with the baseline methods, the defense strategy learned by TSGS has higher utility and greater robustness. Especially, TSGS achieves efficient collaboration during the patrolling detection stage after resource allocation by making full use of real-time information and communication.
computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science