Abstract:Deep cross-modal hashing has achieved excellent retrieval performance with the powerful representation capability of deep neural networks. Regrettably, current methods are inevitably vulnerable to adversarial attacks, especially well-designed subtle perturbations that can easily fool deep cross-modal hashing models into returning irrelevant or the attacker’s specified results. Although adversarial attacks have attracted increasing attention, there are few studies on specialized attacks against deep cross-modal hashing. To solve these issues, we propose a targeted adversarial attack method against deep cross-modal hashing retrieval in this paper. To the best of our knowledge, this is the first work in this research field. Concretely, we first build a progressive fusion module to extract fine-grained target semantics through a progressive attention mechanism. Meanwhile, we design a semantic adaptation network to generate the target prototype code and reconstruct the category label, thus realizing the semantic interaction between the target semantics and the implicit semantics of the attacked model. To bridge modality gaps and preserve local example details, a semantic translator seamlessly translates the target semantics and then embeds them into benign examples in collaboration with a U-Net framework. Moreover, we construct a discriminator for adversarial training, which enhances the visual realism and category discrimination of adversarial examples, thus improving their targeted attack performance. Extensive experiments on widely tested cross-modal retrieval datasets demonstrate the superiority of our proposed method. Also, transferable attacks show that our generated adversarial examples have well generalization capability on targeted attacks. The source codes and datasets are available at https://github.com/tswang0116/TA-DCH.

Cross-Task Attack: A Self-Supervision Generative Framework Based on Attention Shift

Enhancing Cross-task Transferability of Adversarial Examples Via Spatial and Channel Attention

Stealthy Multi-Task Adversarial Attacks

Attention distraction with gradient sharpening for multi-task adversarial attack

CT-GAT: Cross-Task Generative Adversarial Attack Based on Transferability

One-Shot Adversarial Attacks on Visual Tracking With Dual Attention

CDTA: A Cross-Domain Transfer-Based Attack with Contrastive Learning.

Towards cross-task universal perturbation against black-box object detectors in autonomous driving

Targeted Adversarial Attack against Deep Cross-modal Hashing Retrieval

Multi-Task Models Adversarial Attacks

Generating Transferable Adversarial Examples Against Vision Transformers

Robust Superpixel-Guided Attentional Adversarial Attack

Adversarial Attacks on Hidden Tasks in Multi-Task Learning

Using Multiple Self-Supervised Tasks Improves Model Robustness

Attention, Please! Adversarial Defense via Activation Rectification and Preservation

Cross-shaped Adversarial Patch Attack

UCG: A Universal Cross-Domain Generator for Transferable Adversarial Examples

Cross-Modality Attack Boosted by Gradient-Evolutionary Multiform Optimization

Challenging AI+Camera Systems with Physical Adversarial Attacks

VQAttack: Transferable Adversarial Attacks on Visual Question Answering via Pre-trained Models

Boosting Cross-Task Adversarial Attack with Random Blur