Social Media Authentication and Combating Deepfakes using Semi-fragile Invisible Image Watermarking

Aakash Varma Nadimpalli,Ajita Rattani

2024-10-03

Abstract:With the significant advances in deep generative models for image and video synthesis, Deepfakes and manipulated media have raised severe societal concerns. Conventional machine learning classifiers for deepfake detection often fail to cope with evolving deepfake generation technology and are susceptible to adversarial attacks. Alternatively, invisible image watermarking is being researched as a proactive defense technique that allows media authentication by verifying an invisible secret message embedded in the image pixels. A handful of invisible image watermarking techniques introduced for media authentication have proven vulnerable to basic image processing operations and watermark removal attacks. In response, we have proposed a semi-fragile image watermarking technique that embeds an invisible secret message into real images for media authentication. Our proposed watermarking framework is designed to be fragile to facial manipulations or tampering while being robust to benign image-processing operations and watermark removal attacks. This is facilitated through a unique architecture of our proposed technique consisting of critic and adversarial networks that enforce high image quality and resiliency to watermark removal efforts, respectively, along with the backbone encoder-decoder and the discriminator networks. Thorough experimental investigations on SOTA facial Deepfake datasets demonstrate that our proposed model can embed a $64$-bit secret as an imperceptible image watermark that can be recovered with a high-bit recovery accuracy when benign image processing operations are applied while being non-recoverable when unseen Deepfake manipulations are applied. In addition, our proposed watermarking technique demonstrates high resilience to several white-box and black-box watermark removal attacks. Thus, obtaining state-of-the-art performance.

Computer Vision and Pattern Recognition,Artificial Intelligence,Cryptography and Security,Machine Learning,Multimedia

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are media authentication on social media and the problem of combating Deepfake. Specifically:

1. **Media Authentication**: With the progress of generative model technologies, Deepfake and other media - manipulating technologies have caused serious social problems. Traditional machine - learning classifiers perform poorly in detecting Deepfake and are vulnerable to adversarial attacks. Therefore, a new method is required to verify the authenticity and integrity of digital media, especially in the social media environment.

2. **Combating Deepfake**: Deepfake technology can be used to create highly realistic false content, such as involuntary pornography, spreading rumors, etc., which pose a threat to personal privacy, reputation, and social stability. The paper proposes a semi - fragile invisible image watermarking technique to address these issues.

### Specific Objectives of the Paper:
- **Embedding Invisible Watermarks**: Media authentication is achieved by embedding an invisible secret message in a real image. This watermark remains stable under normal image - processing operations (such as compression, color adjustment), but becomes unrecoverable when facial manipulation or Deepfake modification techniques are applied.
- **Improving Robustness**: The designed watermarking framework can resist basic image - processing operations and watermark - removal attacks, while being sensitive to malicious facial transformations (such as Deepfake).
- **Optimizing Model Architecture**: By combining the critic network and the adversarial network, high - image quality and resistance to watermark - removal efforts are ensured. In addition, a backbone encoder - decoder and a discriminative network are also used to improve overall performance.

### Technical Contributions:
1. **A New Semi - Fragile Invisible Facial Image Watermarking Technique**: It provides higher imperceptibility and resistance to adversarial watermark - removal attacks.
2. **Model Imperceptibility Evaluation**: Comparison with the existing state - of - the - art watermarking techniques is made through metrics such as Peak Signal - to - Noise Ratio (PSNR) and Structural Similarity Index (SSIM).
3. **Robustness Analysis**: The robustness of the model to unknown benign and malicious facial manipulations is evaluated, and tests are carried out using different generative models.
4. **Robustness Analysis of Watermark - Removal Attacks**: The robustness of the model to various white - box (such as Fast Gradient Sign Method (FGSM), Carlini & Wagner, Backward Pass Differentiable Approximation (BPDA) and Expectation over Transformations (EOT)) and black - box (based on VAE embedding and reconstruction) watermark - removal attacks is evaluated.
5. **Dataset Evaluation**: Evaluation is carried out on standard datasets (such as FaceForensics++, CelebA and IMDB - WIKI) that are widely used for facial - manipulation Deepfake generation and detection.
6. **Ablation Study**: To better understand the impact of each module (network) in the model and the threat of adversarial attacks to the model.

Through these techniques and methods, the paper aims to provide an effective solution to ensure the authenticity and security of media content on social media, while effectively combating the threats brought by Deepfake.