Abstract:The critical challenge of image inpainting is to infer reasonable semantics and textures for a corrupted image. Typical methods for image inpainting are built upon some prior knowledge to synthesize the complete image. One potential limitation is that those methods often remain undesired blurriness or semantic mistakes in the synthesized image while handling images with large corrupted areas. In this paper, we propose a Collaborative Contrastive Learning-based Generative Model (C2LGM), which learns the content consistency in the same image to ensure that the inferred content of corrupted areas is reasonable compared to the known content by pixel-level reconstruction and high-level semantic reasoning. C2LGM leverages the encoder-decoder based framework to directly learn the mapping from the corrupted image to the intact image and perform the pixel-level reconstruction. To perform semantic reasoning, our C2LGM introduces a Collaborative Contrastive Learning (C2L) mechanism that learns high-level semantic consistency between inferred and known content. Specifically, C2L mechanism introduces the high-frequency edge maps to participate in the process of typical contrastive learning and enables the deep model to ensure the semantic reasonableness between high-frequency structures and pixel-level content by pushing the representations of inferred content and known content close and keeping unrelated semantic content away in the latent feature space. Moreover, C2LGM also directly absorbs the prior knowledge of structural information from the proposed structural spatial attention module, and leverages the texture distribution sampling to improve the quality of synthesized content. As a result, our C2LGM achieves a 0.42 dB improvement over competing methods in terms of the PSNR metric while coping with a % corruption ratio in the Places2 dataset. Extensive experiments - n three benchmark datasets, including Paris Street View, CelebA-HQ, and Places2, demonstrate the advantages of our proposed C2LGM over other state-of-the-art methods for image inpainting both qualitatively and quantitatively.

Image Inpainting Via Conditional Texture and Structure Dual Generation

UCTGAN: Diverse Image Inpainting Based on Unsupervised Cross-Space Translation

Image Inpainting by Bidirectional Information Flow on Texture and Structure

Generative Image Inpainting with Residual Texture Prior and Cross-Layer Contextual Attention

Joint Structure and Texture-guided Image Inpainting with a Multi-Scale Shared Feature Extractor

Image Inpainting Based on Interactive Separation Network and Progressive Reconstruction Algorithm

Image Inpainting Network Based on Deep Fusion of Texture and Structure.

Image Fine-grained Inpainting.

Two-stage structure aware image inpainting based on generative adversarial networks

Image Inpainting: A Contextual Consistent and Deep Generative Adversarial Training Approach

Guiding image inpainting via structure and texture features with dual encoder

Generative Image Inpainting with Residual Attention Learning

Structure First Detail Next: Image Inpainting with Pyramid Generator

Structure-aware Image Inpainting with Two Parallel Streams

Delving Globally into Texture and Structure for Image Inpainting

Bi-encoder Network with Structure-texture Consistency for Image Inpainting.

Image Inpainting Using Wasserstein Generative Adversarial Network

Two-stream coupling network with bidirectional interaction between structure and texture for image inpainting

HSVConnect: HSV guided enhanced content generation network for image inpainting

Collaborative Contrastive Learning-Based Generative Model for Image Inpainting

Image Inpainting Based on Multi-frequency Probabilistic Inference Model