A One-Class Classifier for the Detection of GAN Manipulated Multi-Spectral Satellite Images

Lydia Abady,Giovanna Maria Dimitri,Mauro Barni

2023-05-20

Abstract:The highly realistic image quality achieved by current image generative models has many academic and industrial applications. To limit the use of such models to benign applications, though, it is necessary that tools to conclusively detect whether an image has been generated synthetically or not are developed. For this reason, several detectors have been developed providing excellent performance in computer vision applications, however, they can not be applied as they are to multispectral satellite images, and hence new models must be trained. In general, two-class classifiers can achieve very good detection accuracies, however they are not able to generalise to image domains and generative models architectures different than those used during training. For this reason, in this paper, we propose a one-class classifier based on Vector Quantized Variational Autoencoder 2 (VQ-VAE 2) features to overcome the limitations of two-class classifiers. First, we emphasize the generalization problem that binary classifiers suffer from by training and testing an EfficientNet-B4 architecture on multiple multispectral datasets. Then we show that, since the VQ-VAE 2 based classifier is trained only on pristine images, it is able to detect images belonging to different domains and generated by architectures that have not been used during training. Last, we compare the two classifiers head-to-head on the same generated datasets, highlighting the superiori generalization capabilities of the VQ-VAE 2-based detector.

Computer Vision and Pattern Recognition,Image and Video Processing

What problem does this paper attempt to address?

### Problems Addressed by the Paper

The paper aims to address the problem of detecting Generative Adversarial Network (GAN) manipulated images in multispectral satellite imagery. Specifically, current image generation models can produce highly realistic images, which have broad applications in both academic and industrial fields. However, to restrict these models to benign applications, tools capable of accurately determining whether an image is synthetically generated need to be developed. Existing detectors, although performing well in computer vision tasks, cannot be directly applied to multispectral satellite images, necessitating the training of new models.

### Background and Motivation

1. **Development of Image Generation Technology**: Deep Learning (DL) technologies have made significant progress in various fields, including bioinformatics, computer vision, natural language processing, and object detection. These technologies are also widely used in image forgery and authenticity verification.

2. **Importance of Satellite Images**: Satellite images play a crucial role in numerous applications such as weather forecasting, landscape analysis, agriculture, regional planning, and natural disaster monitoring and detection. With the increase in the number of commercial satellites and the proliferation of high-resolution satellite images, the forgery and manipulation of satellite images have become an increasingly serious issue.

3. **Limitations of Existing Methods**: Existing image forgery detection tools are primarily designed for conventional RGB images and are difficult to apply directly to multispectral satellite images. Multispectral satellite images have different characteristics and properties, often containing more than 3 bands (e.g., Sentinel-2 Level 1C optical images have 13 bands), with each band having a different Ground Sampling Distance (GSD). Additionally, each pixel in multispectral images is typically represented by more than 8 bits per band (Sentinel-2 Level 1C images are 12-bit). These characteristics make it challenging for existing detection tools to be directly applied to multispectral satellite images.

### Research Objectives

1. **Propose a New Detection Method**: The paper proposes a one-class classifier based on Vector Quantized Variational Autoencoder 2 (VQ-VAE 2) features to overcome the generalization limitations of binary classifiers.

2. **Validate the Effectiveness of the New Method**: By evaluating the performance of the one-class classifier on multiple multispectral GAN-synthesized Sentinel-2 Level 1C satellite datasets and comparing it with traditional EfficientNet-B4 based detectors, the paper demonstrates the superior generalization capability of the new method.

### Main Contributions

1. **Design of a One-Class Classifier**: The paper proposes a one-class classifier based on VQ-VAE 2 features, which is trained only on original images and can detect images generated by different domain and unseen generative model architectures.

2. **Validation of Generalization Capability**: Experiments validate the generalization capability of the VQ-VAE 2 classifier across different datasets, particularly when the test dataset does not match the training dataset, showing significant advantages.

3. **Handling of Multispectral Images**: The study investigates the performance of VQ-VAE 2 in handling multispectral satellite images, especially its performance across 13 bands.

### Conclusion

The one-class classifier proposed in the paper, based on VQ-VAE 2 features, performs excellently in detecting GAN-manipulated images in multispectral satellite imagery, particularly in terms of generalization capability, outperforming traditional binary classifiers. This method provides a new solution for forgery detection in multispectral satellite images.