Abstract:Adversarial attacks threaten the reliability of machine learning models in critical applications like autonomous vehicles and defense systems. As object detectors become more robust with models like YOLOv8, developing effective adversarial methodologies is increasingly challenging. We present Truck Adversarial Camouflage Optimization (TACO), a novel framework that generates adversarial camouflage patterns on 3D vehicle models to deceive state-of-the-art object detectors. Adopting Unreal Engine 5, TACO integrates differentiable rendering with a Photorealistic Rendering Network to optimize adversarial textures targeted at YOLOv8. To ensure the generated textures are both effective in deceiving detectors and visually plausible, we introduce the Convolutional Smooth Loss function, a generalized smooth loss function. Experimental evaluations demonstrate that TACO significantly degrades YOLOv8's detection performance, achieving an AP@0.5 of 0.0099 on unseen test data. Furthermore, these adversarial patterns exhibit strong transferability to other object detection models such as Faster R-CNN and earlier YOLO versions.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to generate effective adversarial camouflage patterns to deceive the state - of - the - art object detection models, especially YOLOv8, so that a specific truck model becomes undetectable in these detection models. With the development of object detection technology, especially the increasing robustness of models such as YOLOv8, developing effective adversarial methods has become more and more challenging. Therefore, this paper proposes the Truck Adversarial Camouflage Optimization (TACO) framework, aiming to deceive these advanced object detection models by generating adversarial camouflage patterns. ### Specific problem description 1. **Improve the effectiveness of adversarial attacks**: Traditional adversarial attack methods have limited application effects in the physical world, especially under complex lighting, angle and environmental conditions. The TACO framework ensures that the generated adversarial textures can not only deceive the detector but also be visually reasonable by introducing photo - realistic rendering techniques and differentiable rendering pipelines. 2. **Attacks against the latest models**: Most of the existing adversarial attack research focuses on older versions of object detection models (such as YOLOv3 or Faster R - CNN), and the performance and robustness of these models may be inferior to the latest YOLOv8. Therefore, the TACO framework is specifically optimized for YOLOv8 to verify its effectiveness on the latest model. 3. **Ensure the smoothness and visual rationality of adversarial textures**: In order to ensure that the generated adversarial textures do not appear too abrupt or unnatural in practical applications, TACO introduces the Convolutional Smooth Loss function, making the generated textures both effective and visually reasonable. ### Solution overview - **Use Unreal Engine 5 for photo - realistic rendering**: TACO utilizes the powerful rendering capabilities of UE5 to generate realistic 3D vehicle model images and optimize adversarial textures through differentiable rendering pipelines. - **Design new loss functions**: In order to ensure that the generated textures can deceive the detector and be visually reasonable, TACO introduces the Convolutional Smooth Loss function and combines class confidence loss and IoU loss in the optimization process. - **Experimental verification**: Through a large number of experimental verifications, the TACO framework significantly reduces the detection performance of YOLOv8, achieving a low detection rate of AP@0.5 = 0.0099 on unseen test data, and these adversarial patterns also show strong transferability to other object detection models (such as Faster R - CNN and early YOLO versions). ### Summary of mathematical formulas 1. **Generation of enhanced truck images**: \[ X_{\text{enh}} = R(\text{Mesh}, T_{\text{adv}}, \theta_c, X_d, X_{\text{gray}}) \] 2. **Generation of final adversarial images**: \[ X_{\text{adv}} = X_{\text{enh}} \cdot M+X_{\text{ref}} \cdot (1 - M) \] 3. **Optimization objective**: \[ T^*_{\text{adv}}=\arg\min_{T_{\text{adv}}} L(F(X_{\text{enh}}; \theta_F)) \] 4. **Class confidence loss**: \[ L_{\text{cls}}=-C\sum_{c = 1}\sum_{i\in\Omega_{\text{iop}}}\log(1 - b^{i,c}_{\text{cls}}) \] 5. **Intersection over Union loss**

TACO: Adversarial Camouflage Optimization on Trucks to Fool Object Detectors

Adversarial Robustness Analysis of LiDAR-included Models in Autonomous Driving

DTA: Physical Camouflage Attacks using Differentiable Transformation Network

Adversarial Attack and Defense of YOLO Detectors in Autonomous Driving Scenarios

Learning Coated Adversarial Camouflages for Object Detectors

Camouflaged Adversarial Patch Attack on Object Detector

Building Towards "Invisible Cloak": Robust Physical Adversarial Attack on YOLO Object Detector

Entropy-Boosted Adversarial Patch for Concealing Pedestrians in YOLO Models

The Weaknesses of Adversarial Camouflage in Overhead Imagery

FCA: Learning a 3D Full-Coverage Vehicle Camouflage for Multi-View Physical Adversarial Attack

CNCA: Toward Customizable and Natural Generation of Adversarial Camouflage for Vehicle Detectors

Misleading attention and classification: An adversarial attack to fool object detection models in the real world

To Make Yourself Invisible with Adversarial Semantic Contours

Transient Adversarial 3D Projection Attacks on Object Detection in Autonomous Driving

CamoNet: A Target Camouflage Network for Remote Sensing Images Based on Adversarial Attack

TOG: Targeted Adversarial Objectness Gradient Attacks on Real-time Object Detection Systems

Developing Imperceptible Adversarial Patches to Camouflage Military Assets From Computer Vision Enabled Technologies

Searching Realistic-Looking Adversarial Objects For Autonomous Driving Systems

RAUCA: A Novel Physical Adversarial Attack on Vehicle Detectors via Robust and Accurate Camouflage Generation

You Cannot Easily Catch Me: A Low-Detectable Adversarial Patch for Object Detectors

Adversarial defenses for object detectors based on Gabor convolutional layers