Jin Hu,Xianglong Liu,Jiakai Wang,Junkai Zhang,Xianqi Yang,Haotong Qin,Yuqing Ma,Ke Xu
Abstract:Physical adversarial examples (PAEs) are regarded as "whistle-blowers" of real-world risks in deep-learning applications. However, current PAE generation studies show limited adaptive attacking ability to diverse and varying scenes. The key challenges in generating dynamic PAEs are exploring their patterns under noisy gradient feedback and adapting the attack to agnostic scenario natures. To address the problems, we present DynamicPAE, the first generative framework that enables scene-aware real-time physical attacks beyond static attacks. Specifically, to train the dynamic PAE generator under noisy gradient feedback, we introduce the residual-driven sample trajectory guidance technique, which redefines the training task to break the limited feedback information restriction that leads to the degeneracy problem. Intuitively, it allows the gradient feedback to be passed to the generator through a low-noise auxiliary task, thereby guiding the optimization away from degenerate solutions and facilitating a more comprehensive and stable exploration of feasible PAEs. To adapt the generator to agnostic scenario natures, we introduce the context-aligned scene expectation simulation process, consisting of the conditional-uncertainty-aligned data module and the skewness-aligned objective re-weighting module. The former enhances robustness in the context of incomplete observation by employing a conditional probabilistic model for domain randomization, while the latter facilitates consistent stealth control across different attack targets by automatically reweighting losses based on the skewness indicator. Extensive digital and physical evaluations demonstrate the superior attack performance of DynamicPAE, attaining a 1.95 $\times$ boost (65.55% average AP drop under attack) on representative object detectors (e.g., Yolo-v8) over state-of-the-art static PAE generating methods.
What problem does this paper attempt to address?
### What problems does this paper attempt to solve?
This paper aims to solve the problem of generating dynamic physical adversarial examples (Dynamic PAEs). Specifically, the author focuses on how to generate physical adversarial examples that adapt to different scenarios in real - time in the real world, in order to address the shortcomings of current static PAE generation methods in diversity and changing scenarios.
#### Background and problem description
1. **Importance of physical adversarial examples**:
- Physical adversarial examples (PAEs) are regarded as "whistleblowers" to reveal the potential risks of deep learning applications in the real world, so they are worthy of further study.
- Current PAE generation research shows limited ability in adapting to diverse and changing scenarios, which reveals the need for real - time generation of dynamic PAEs.
2. **Limitations of existing methods**:
- Existing PAE generation methods usually regard the problem as a static problem. They either try to generalize PAE in all simulated physical scenarios or need to be retrained every time to adapt to a new environment. This results in insufficient adaptability and generation efficiency.
- Dynamic PAEs have not been fully studied. Related attempts are limited to determining patch locations, simulating and controlling a few states, or optimizing clustering scenarios in the laboratory, and have not achieved a truly dynamic response.
#### Core challenges of the paper
1. **Noise gradient feedback in exploration mode**:
- In attack training, noise gradient feedback (such as randomly injected noise) hinders the effective exploration of potentially sparse - distributed PAEs, leading to optimization degradation and infinite gradient problems.
2. **Adapting to the nature of unknown scenarios**:
- Attack scenarios in the real world contain unknown attributes, including incomplete observations and different attack targets, which affect the balance between targets. Therefore, it is necessary to model scene - aware attacks to eliminate the physical - digital gap and maintain consistent generation behavior.
#### Solutions
To solve the above problems, the author proposes the DynamicPAE framework, which mainly includes the following two key technologies:
1. **Residual - Driven Sample Trajectory Guidance**:
- By introducing a low - noise auxiliary task, the training task is redefined so that the gradient feedback can be transmitted to the generator, thereby guiding the optimization away from degenerate solutions and promoting more comprehensive and stable PAE exploration.
- Formula representation:
\[
L_{\lambda}(L_{Atk}, L_R):=\lambda L_{Atk}(\delta(·, \lambda))+(1 - \lambda) L_R(\delta(·, \lambda))
\]
where \( L_R \) is the auxiliary task loss and \( \lambda \) is a weight parameter.
2. **Context - Aligned Scene Expectation Simulation**:
- It includes a Conditional - Uncertainty - Aligned Data Module and a Skewness - Aligned Objective Re - weighting Module.
- The former enhances robustness under incomplete observations through a conditional probability model, and the latter achieves consistent stealth control between different attack targets by automatically adjusting the loss weights.
#### Experimental verification
Through extensive digital and physical experiments, DynamicPAE has demonstrated superior attack performance, with an average AP drop of 65.55% and an inference delay of only 12ms. These results show that DynamicPAE outperforms existing PAE generation methods in multiple settings.
In summary, this paper solves the core challenges in dynamic PAE generation, that is, generating effective adversarial examples in an uncertain environment, by proposing the DynamicPAE framework, and realizes end - to - end dynamic PAE modeling.