Image Forgery Localization via Guided Noise and Multi-Scale Feature Aggregation

Yakun Niu,Pei Chen,Lei Zhang,Lei Tan,Yingjian Chen

2024-11-17

Abstract:Image Forgery Localization (IFL) technology aims to detect and locate the forged areas in an image, which is very important in the field of digital forensics. However, existing IFL methods suffer from feature degradation during training using multi-layer convolutions or the self-attention mechanism, and perform poorly in detecting small forged regions and in robustness against post-processing. To tackle these, we propose a guided and multi-scale feature aggregated network for IFL. Spectifically, in order to comprehensively learn the noise feature under different types of forgery, we develop an effective noise extraction module in a guided way. Then, we design a Feature Aggregation Module (FAM) that uses dynamic convolution to adaptively aggregate RGB and noise features over multiple scales. Moreover, we propose an Atrous Residual Pyramid Module (ARPM) to enhance features representation and capture both global and local features using different receptive fields to improve the accuracy and robustness of forgery localization. Expensive experiments on 5 public datasets have shown that our proposed model outperforms several the state-of-the-art methods, specially on small region forged image.

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

This paper attempts to solve several key problems in Image Forgery Localization (IFL) technology:

1. **Feature Degradation Problem**: When using multi - layer convolution or multi - head self - attention mechanism for training, existing IFL methods will encounter feature degradation, resulting in a decline in model performance.
2. **Difficulty in Detecting Small Forged Areas**: Existing methods perform poorly in detecting and locating small forged areas.
3. **Insufficient Robustness to Post - processing**: Existing methods have difficulty maintaining high detection accuracy when facing post - processing operations such as image compression.

To solve these problems, the author proposes a network architecture based on guided noise and multi - scale feature aggregation. Specifically, this method includes the following innovations:

- **Guided Noise Extraction Module**: By combining the guided filter and Sobel filter to enhance edge information and extract different types of forgery traces.
- **Feature Aggregation Module (FAM)**: Use dynamic convolution to adaptively aggregate RGB and noise features, thereby improving feature expression ability.
- **Atrous Residual Pyramid Module (ARPM)**: Expand the receptive field through atrous convolution, capture global and local features, alleviate the feature degradation problem, and improve the localization accuracy of the forged area.

### Main Contributions

1. **Novel Network Architecture**: A network architecture based on guided noise and multi - scale feature aggregation is proposed, which can learn different types of forgery traces more effectively.
2. **Feature Aggregation Module (FAM)**: By fusing RGB and noise features through dynamic convolution, the feature expression ability is improved.
3. **Atrous Residual Pyramid Module (ARPM)**: By expanding the receptive field through atrous convolution, features at different scales are learned, and the detection ability for small forged areas is improved.
4. **Experimental Verification**: The experimental results on 5 public datasets show that this method is superior to existing IFL methods, especially in the detection of small forged areas.

### Formula Summary

1. **Image Decomposition Formula**:
\[
I = I_c+I_f
\]
where \(I\) is the input image, \(I_c\) is the content information, and \(I_f\) is the forgery information.

2. **Forgery Information Extraction Formula**:
\[
I_f=\vert I - \text{Guide}(I)\vert
\]
The forgery information is extracted through the guided filter.

3. **Guided Noise Extraction Formula**:
\[
I_g = I_f+I_s = I_f+\text{Sobel}(I)
\]
The edge information is further extracted by combining the Sobel filter.

4. **Feature Aggregation Module (FAM) Formula**:
\[
\hat{f}_i^{\text{rgb}} = C_{5\times5}(\text{CD}(C_{1\times1}(\text{Sobel}(f_i^{\text{rgb}})))) + f_i^{\text{rgb}}
\]
\[
\hat{f}_i^n = C_{7\times7}(\text{Max}(C_{1\times1}(f_i^n))) + f_i^n
\]
\[
f_i^{\text{Agg}}=\text{ReLU}(\text{BN}(C_{1\times1}(\hat{f}_i^{\text{rgb}}:\hat{f}_i^n)))
\]

5. **Atrous Residual Pyramid Module (ARPM) Formula**:
\[
f_{\text{avg}} = C_{1\times1}(\text{GAP}(f_{\text{Agg}}))
\]
\[
f_{1\times1}=C_{1\times1}
\]