Abstract:Image harmonization task aims at harmonizing different composite foreground regions according to specific background image. Previous methods would rather focus on improving the reconstruction ability of the generator by some internal enhancements such as attention, adaptive normalization and light adjustment, $etc.$. However, they pay less attention to discriminating the foreground and background appearance features within a restricted generator, which becomes a new challenge in image harmonization task. In this paper, we propose a novel image harmonization framework with external style fusion and region-wise contrastive learning scheme. For the external style fusion, we leverage the external background appearance from the encoder as the style reference to generate harmonized foreground in the decoder. This approach enhances the harmonization ability of the decoder by external background guidance. Moreover, for the contrastive learning scheme, we design a region-wise contrastive loss function for image harmonization task. Specifically, we first introduce a straight-forward samples generation method that selects negative samples from the output harmonized foreground region and selects positive samples from the ground-truth background region. Our method attempts to bring together corresponding positive and negative samples by maximizing the mutual information between the foreground and background styles, which desirably makes our harmonization network more robust to discriminate the foreground and background style features when harmonizing composite images. Extensive experiments on the benchmark datasets show that our method can achieve a clear improvement in harmonization quality and demonstrate the good generalization capability in real-scenario applications.

Intrinsic Image Harmonization

Intrinsic Harmonization for Illumination-Aware Compositing

Deep Image Harmonization with Globally Guided Feature Transformation and Relation Distillation

Deep Image-based Illumination Harmonization

Diverse Image Harmonization

Relightful Harmonization: Lighting-aware Portrait Background Replacement

DiffHarmony: Latent Diffusion Model Meets Image Harmonization

CcHarmony: Color-checker based Image Harmonization Dataset

SIDNet: Learning Shading-aware Illumination Descriptor for Image Harmonization

Deep Image Harmonization by Bridging the Reality Gap

NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo

Dense Pixel-to-Pixel Harmonization via Continuous Image Representation

Spatially-Varying Illumination-Aware Indoor Harmonization

Image Harmonization Dataset Iharmony4: HCOCO, HAdobe5k, HFlickr, and Hday2night

Harmonizer: Learning to Perform White-Box Image and Video Harmonization

FRIH: Fine-grained Region-aware Image Harmonization

SSH: A Self-Supervised Framework for Image Harmonization

Painterly Image Harmonization in Dual Domains

Deep Image Harmonization in Dual Color Spaces

Image Harmonization with Region-wise Contrastive Learning

DoveNet: Deep Image Harmonization via Domain Verification