Abstract:The critical challenge of image inpainting is to infer reasonable semantics and textures for a corrupted image. Typical methods for image inpainting are built upon some prior knowledge to synthesize the complete image. One potential limitation is that those methods often remain undesired blurriness or semantic mistakes in the synthesized image while handling images with large corrupted areas. In this paper, we propose a Collaborative Contrastive Learning-based Generative Model (C2LGM), which learns the content consistency in the same image to ensure that the inferred content of corrupted areas is reasonable compared to the known content by pixel-level reconstruction and high-level semantic reasoning. C2LGM leverages the encoder-decoder based framework to directly learn the mapping from the corrupted image to the intact image and perform the pixel-level reconstruction. To perform semantic reasoning, our C2LGM introduces a Collaborative Contrastive Learning (C2L) mechanism that learns high-level semantic consistency between inferred and known content. Specifically, C2L mechanism introduces the high-frequency edge maps to participate in the process of typical contrastive learning and enables the deep model to ensure the semantic reasonableness between high-frequency structures and pixel-level content by pushing the representations of inferred content and known content close and keeping unrelated semantic content away in the latent feature space. Moreover, C2LGM also directly absorbs the prior knowledge of structural information from the proposed structural spatial attention module, and leverages the texture distribution sampling to improve the quality of synthesized content. As a result, our C2LGM achieves a 0.42 dB improvement over competing methods in terms of the PSNR metric while coping with a % corruption ratio in the Places2 dataset. Extensive experiments - n three benchmark datasets, including Paris Street View, CelebA-HQ, and Places2, demonstrate the advantages of our proposed C2LGM over other state-of-the-art methods for image inpainting both qualitatively and quantitatively.

Contrastive local constraint for irregular image reconstruction and editability

Contrastive Learning for Local and Global Learning MRI Reconstruction

How does Contrastive Learning Organize Images?

Cycle Contrastive Adversarial Learning for Unsupervised image Deraining

Cycle contrastive adversarial learning with structural consistency for unsupervised high-quality image deraining transformer

Consistency-Contrast Learning for Conceptual Coding

Learning from History: Task-agnostic Model Contrastive Learning for Image Restoration

High-Fidelity GAN Inversion for Image Attribute Editing

Understanding and Generalizing Contrastive Learning from the Inverse Optimal Transport Perspective.

Local Structure-aware Graph Contrastive Representation Learning

Collaborative Contrastive Learning-Based Generative Model for Image Inpainting

Contrastive Learning Via Equivariant Representation

Predicting the Invariance Behind Residuals: A Novel GAN Inversion Method for Image Editing and Detail Retaining

Unsupervised Synthetic Image Refinement via Contrastive Learning and Consistent Semantic-Structural Constraints

Localized Graph Contrastive Learning

Residual Contrastive Learning for Image Reconstruction: Learning Transferable Representations from Noisy Images

Adaptive Multi-head Contrastive Learning

Interventional Contrastive Learning with Meta Semantic Regularizer

Contrastive Attraction and Contrastive Repulsion for Representation Learning

In-Domain GAN Inversion for Faithful Reconstruction and Editability

CLAMP: Contrastive LAnguage Model Prompt-tuning