Abstract:Traffic sign recognition systems play a crucial role in assisting drivers to make informed decisions while driving. However, due to the heavy reliance on deep learning technologies, particularly for future connected and autonomous driving, these systems are susceptible to adversarial attacks that pose significant safety risks to both personal and public transportation. Notably, researchers recently identified a new attack vector to deceive sign recognition systems: projecting well-designed adversarial light patches onto traffic signs. In comparison with traditional adversarial stickers or graffiti, these emerging light patches exhibit heightened aggression due to their ease of implementation and outstanding stealthiness. To effectively counter this security threat, we propose a universal image inpainting mechanism, namely, SafeSign. It relies on attention-enabled multi-view image fusion to repair traffic signs contaminated by adversarial light patches, thereby ensuring the accurate sign recognition. Here, we initially explore the fundamental impact of malicious light patches on the local and global feature spaces of authentic traffic signs. Then, we design a binary mask-based U-Net image generation pipeline outputting diverse contaminated sign patterns, to provide our image inpainting model with needed training data. Following this, we develop an attention mechanism-enabled neural network to jointly utilize the complementary information from multi-view images to repair contaminated signs. Finally, extensive experiments are conducted to evaluate SafeSign's effectiveness in resisting potential light patch-based attacks, bringing an average accuracy improvement of 54.8% in three widely-used sign recognition models

What problem does this paper attempt to address?

This paper attempts to address the security issues of the Traffic Sign Recognition (TSR) system when facing new types of adversarial light - spot attacks. Specifically, the researchers have discovered a new attack method: by remotely projecting carefully - designed visible or invisible light - spots onto traffic signs to deceive the TSR system, leading to misjudgments. This attack method is highly concealed and easy to implement, posing a serious threat to personal and public traffic safety. To effectively deal with this security threat, the author proposes a general - purpose image inpainting mechanism - SafeSign. This mechanism is based on the multi - view image fusion technology of the attention mechanism and is used to repair traffic sign images contaminated by adversarial light - spots, thereby ensuring accurate sign recognition. The following are the main contributions of the paper: 1. **Proposing the SafeSign mechanism**: SafeSign is a general - purpose defense mechanism designed to resist potential light - spot attacks by repairing contaminated traffic sign images. Its advantages lie in the fact that it does not require retraining of existing recognition models, prior knowledge of attack settings, and does not need to modify the TSR hardware architecture. 2. **Designing a U - Net - based adversarial sign generation model**: Combining binary masks to generate diverse contaminated sign patterns, providing sufficient training samples for the repair model. 3. **Constructing an attention - mechanism - driven sign reconstruction neural network**: Utilizing the SENet network and multi - head self - attention modules to fully exploit the complementary information in multi - view images to repair contaminated sign images and improve the detection reliability of the TSR system. 4. **Extensive experimental verification**: Through a large number of experiments on public datasets and widely - used recognition models, the effectiveness of SafeSign has been verified. The experimental results show that SafeSign can significantly improve the average recognition accuracy rate and precision, increasing them by 54.8% and 58.5% respectively, and can effectively resist four representative light - spot attacks (infrared, laser, artificial shadow, and projected graffiti). In conclusion, as an algorithm - level plug - in solution, SafeSign can provide a general - purpose defense mechanism to resist various possible light - spot attacks without incurring additional costs, ensuring the security of the traffic sign recognition system.

Secure Traffic Sign Recognition: An Attention-Enabled Universal Image Inpainting Mechanism against Light Patch Attacks

An Efficient Framework for Detection and Recognition of Numerical Traffic Signs

A defense method based on attention mechanism against traffic sign adversarial samples

A Three-Stage Real-Time Detector for Traffic Signs in Large Panoramas

Traffic Sign Attack Via Pinpoint Region Probability Estimation Network

Targeted Attention Attack on Deep Learning Models in Road Sign Recognition

A Traffic Sign Recognition Method Under Complex Illumination Conditions

Targeted Physical-World Attention Attack on Deep Learning Models in Road Sign Recognition

Traffic Sign Recognition from Digital Images by Using Deep Learning

A feature‐enhanced hybrid attention network for traffic sign recognition in real scenes

Invisible Optical Adversarial Stripes on Traffic Sign against Autonomous Vehicles

Road Traffic Sign Detection Method Based on RTS R-CNN Instance Segmentation Network

ITPatch: An Invisible and Triggered Physical Adversarial Patch against Traffic Sign Recognition

Adversarial Attacks on Traffic Sign Recognition: A Survey

The Improved Framework for Traffic Sign Recognition Using Guided Image Filtering

Sample selection of adversarial attacks against traffic signs

A Hybrid Defense Method against Adversarial Attacks on Traffic Sign Classifiers in Autonomous Vehicles

SSIM-Based Autoencoder Modeling to Defeat Adversarial Patch Attacks

VSSA-NET: Vertical Spatial Sequence Attention Network for Traffic Sign Detection

Traffic Sign Recognition With Lightweight Two-Stage Model in Complex Scenes

Traffic-Sign Detection and Classification in the Wild