Deep Image Composition Meets Image Forgery

Eren Tahir,Mert Bal

2024-04-26

Abstract:Image forgery is a topic that has been studied for many years. Before the breakthrough of deep learning, forged images were detected using handcrafted features that did not require training. These traditional methods failed to perform satisfactorily even on datasets much worse in quality than real-life image manipulations. Advances in deep learning have impacted image forgery detection as much as they have impacted other areas of computer vision and have improved the state of the art. Deep learning models require large amounts of labeled data for training. In the case of image forgery, labeled data at the pixel level is a very important factor for the models to learn. None of the existing datasets have sufficient size, realism and pixel-level labeling at the same time. This is due to the high cost of producing and labeling quality images. It can take hours for an image editing expert to manipulate just one image. To bridge this gap, we automate data generation using image composition techniques that are very related to image forgery. Unlike other automated data generation frameworks, we use state of the art image composition deep learning models to generate spliced images close to the quality of real-life manipulations. Finally, we test the generated dataset on the SOTA image manipulation detection model and show that its prediction performance is lower compared to existing datasets, i.e. we produce realistic images that are more difficult to detect. Dataset will be available at

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper attempts to address the issue of insufficient datasets in image forgery detection. Specifically, existing image forgery datasets are lacking in terms of quantity, authenticity, and pixel-level annotations, which limits the performance improvement of deep learning models in image forgery detection. To solve these problems, the authors propose a method for automatically generating image splicing datasets, using image synthesis techniques to create forged images that are close to the quality of real operations. Through this method, the authors aim to improve the authenticity of the generated images, making them more difficult for existing image manipulation detection models to recognize, thereby advancing the development of image forgery detection technology.