Abstract:Cyber-Physical Systems (CPSs) are increasingly prevalent across various industrial and daily-life domains, with applications ranging from robotic operations to autonomous driving. With recent advancements in artificial intelligence (AI), learning-based components, especially AI controllers, have become essential in enhancing the functionality and efficiency of CPSs. However, the lack of interpretability in these AI controllers presents challenges to the safety and quality assurance of AI-enabled CPSs (AI-CPSs). Existing methods for improving the safety of AI controllers often involve neural network repair, which requires retraining with additional adversarial examples or access to detailed internal information of the neural network. Hence, these approaches have limited applicability for black-box policies, where only the inputs and outputs are accessible during operation. To overcome this, we propose MORTAR, a runtime action repair framework designed for AI-CPSs in this work. MORTAR begins by constructing a prediction model that forecasts the quality of actions proposed by the AI controller. If an unsafe action is detected, MORTAR then initiates a repair process to correct it. The generation of repaired actions is achieved through an optimization process guided by the safety estimates from the prediction model. We evaluate the effectiveness of MORTAR across various CPS tasks and AI controllers. The results demonstrate that MORTAR can efficiently improve task completion rates of AI controllers under specified safety specifications. Meanwhile, it also maintains minimal computational overhead, ensuring real-time operation of the AI-CPSs.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: How to improve the security and reliability of AI - enabled Cyber - Physical Systems (AI - CPSs) based on Deep Reinforcement Learning (DRL) controllers, especially in the case of black - box policies. Specifically, existing methods usually rely on neural network repair, which requires retraining or access to the internal information of the neural network, but this method is not applicable to black - box policies and has high computational cost and operational complexity in practical applications.
To solve these problems, the paper proposes a framework named MORTAR (Model - based Runtime Action Repair Framework for AI - enabled Cyber - Physical Systems). The main goal of MORTAR is to correct the unsafe actions generated by the AI controller at runtime, thereby ensuring the security of the system. The following are the specific solutions of MORTAR:
1. **Prediction Model Construction**:
- MORTAR first constructs a prediction model by collecting safety data of different state - action pairs.
- This prediction model is used to estimate whether the actions generated by the DRL controller will violate safety specifications in the future. Specifically, it uses Signal Temporal Logic (STL) scores to evaluate the safety of actions.
2. **Real - time Action Correction**:
- At runtime, MORTAR uses the prediction model to monitor the actions generated by the AI controller in real - time.
- If an unsafe action is detected, MORTAR will initiate an optimization process to generate a new, safer action. This process is achieved through gradient optimization techniques, ensuring that the new action not only meets the safety requirements but also is as close as possible to the original action to maintain the continuity and stability of the task.
3. **Experimental Verification**:
- The paper conducts large - scale experiments through multiple CPS tasks and different DRL controllers to verify the effectiveness of MORTAR.
- The experimental results show that MORTAR can not only significantly improve the task completion rate but also maintain low computational overhead while ensuring safety, ensuring real - time operation.
In summary, MORTAR provides a novel and efficient method that can correct the unsafe actions generated by the AI controller in real - time without relying on detailed neural network information, thereby improving the security and reliability of AI - CPSs. This makes MORTAR particularly suitable for application scenarios that require high security, such as autonomous driving, robotic operations, etc.
Formula part:
- The calculation formula of the STL score is \( \text{ROB}(\zeta, \varphi) \), where \( \zeta \) is the system output trajectory and \( \varphi \) is the safety specification.
- The calculation formula of the maximum STL score \( \psi_{\text{max}} \) is \( \psi_{\text{max}}=\bar{\psi}+ 2\sigma_\psi \), where \( \bar{\psi} \) and \( \sigma_\psi \) are the mean and standard deviation of the STL scores of all trajectories in the training dataset respectively.
- The optimization problem of action patching is defined as follows:
\[
\min_{a_p^t}\|M(s_t, a_t + a_p^t)-\psi_{\text{max}}\|_2
\]
With the constraints:
\[
\begin{aligned}
&M(s_t, a_t + a_p^t)\geq\psi_{\text{thres}}\\
&M(s_t, a_t + a_p^t)\leq\psi_{\text{max}}\\
&A_{\text{min}}\leq a_t + a_p^t\leq A_{\text{max}}
\end{aligned}
\]
Where the calculation formulas of \( A_{\text{min}} \) and \( A_{\text{max}} \) are respectively: