Resilient control under denial-of-service and uncertainty: An adaptive dynamic programming approach

Weinan Gao,Zhong-Ping Jiang,Tianyou Chai
2024-11-11
Abstract:In this paper, a new framework for the resilient control of continuous-time linear systems under denial-of-service (DoS) attacks and system uncertainty is presented. Integrating techniques from reinforcement learning and output regulation theory, it is shown that resilient optimal controllers can be learned directly from real-time state and input data collected from the systems subjected to attacks. Sufficient conditions are given under which the closed-loop system remains stable given any upper bound of DoS attack duration. Simulation results are used to demonstrate the efficacy of the proposed learning-based framework for resilient control under DoS attacks and model uncertainty.
Systems and Control
What problem does this paper attempt to address?
This paper attempts to solve the problem of resilient control for continuous - time linear systems under the conditions of denial - of - service (DoS) attacks and system uncertainties. Specifically, the authors propose a new framework based on adaptive dynamic programming (ADP), which directly learns the resilient optimal controller through real - time state and input data to ensure that the system can still maintain stability and achieve output regulation when under attack. ### Main Problems and Contributions 1. **Problem Description**: - Modern control systems face the problem of increasing complexity and uncertainty, especially when suffering from DoS attacks. - Traditional control methods rely on accurate system models, but in practical applications, obtaining an accurate system model may be difficult or impossible. - It is necessary to design a controller that can resist DoS attacks and maintain stability without an accurate system model. 2. **Research Objectives**: - Develop a learning - based resilient optimal control framework so that the system can still maintain stability and achieve output regulation in the presence of DoS attacks and uncertainties. - The proposed method does not require accurate knowledge of system dynamics, but directly learns the control strategy through online data. 3. **Main Contributions**: - **New Framework**: A new learning framework is proposed, which combines reinforcement learning and output regulation theory for designing resilient optimal controllers. - **Stability Assurance**: Sufficient conditions for the stability of the closed - loop system are given under the condition of the upper limit of the DoS attack duration. - **Data - Driven Method**: This framework is data - based and is applicable to cases where the system dynamics are unknown. - **Algorithm Development**: A specific algorithm is proposed to implement the above framework, and its effectiveness is verified through simulation. ### Method Overview - **System Model**: Consider a class of continuous - time linear systems affected by DoS attacks, described as: \[ \dot{x}(t)=Ax(t)+Bu(t)+Dv(t) \] \[ e(t)=Cx(t)+Fv(t) \] where \(u(t)\in\mathbb{R}\) is the control input, \(x(t)\in\mathbb{R}^n\) is the state, and \(v(t)\in\mathbb{R}^q\) is the state of the exogenous system. - **DoS Attack Model**: Define the time interval of DoS attacks and assume that the attack frequency and duration satisfy certain conditions (such as Assumption 3 and Assumption 4). - **Resilient Optimal Control**: Through the adaptive dynamic programming method, learn an optimal control strategy that can maintain stability and achieve output regulation under DoS attacks. The specific steps include: 1. Estimate the divergence rate of the system under DoS attacks. 2. Determine the desired convergence rate of the closed - loop system during normal communication according to the estimated divergence rate and the lower limit of the DoS attack duration. 3. Use the ADP method to learn the resilient optimal control strategy. ### Experimental Verification The effectiveness of the proposed resilient control method is verified through a case study of adaptive cruise control. The experimental results show that this method outperforms the other two traditional control methods (internal model controller and model - based resilient controller) during DoS attacks, especially in terms of transient response and steady - state performance. ### Conclusion This paper proposes a learning - based resilient optimal control framework, which successfully solves the control problem of continuous - time linear systems under the conditions of DoS attacks and uncertainties. This method can not only design a controller without an accurate system model, but also ensure the stability and output regulation performance of the system.