WSSL: Weighted Self-supervised Learning Framework For Image-inpainting

Shubham Gupta,Rahul Kunigal Ravishankar,Madhoolika Gangaraju,Poojasree Dwarkanath,Natarajan Subramanyam

2023-08-25

Abstract:Image inpainting is the process of regenerating lost parts of the image. Supervised algorithm-based methods have shown excellent results but have two significant drawbacks. They do not perform well when tested with unseen data. They fail to capture the global context of the image, resulting in a visually unappealing result. We propose a novel self-supervised learning framework for image-inpainting: Weighted Self-Supervised Learning (WSSL) to tackle these problems. We designed WSSL to learn features from multiple weighted pretext tasks. These features are then utilized for the downstream task, image-inpainting. To improve the performance of our framework and produce more visually appealing images, we also present a novel loss function for image inpainting. The loss function takes advantage of both reconstruction loss and perceptual loss functions to regenerate the image. Our experimentation shows WSSL outperforms previous methods, and our loss function helps produce better results.

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

The paper primarily aims to address two key issues in the field of image inpainting:

1. **Performance degradation when handling unseen data with existing methods**: Supervised learning algorithms often perform poorly when dealing with data that did not appear in the training set.
2. **Difficulty in capturing the overall contextual information of images**: Existing methods, especially when repairing larger missing areas, often fail to capture the overall structure and contextual information of the image, resulting in poor visual effects of the repaired image.

To tackle the above challenges, the authors propose a new self-supervised learning framework—Weighted Self-Supervised Learning (WSSL). WSSL aims to learn image features through multiple weighted pre-training tasks, which are then used for the image inpainting task. Additionally, the paper introduces a new loss function that combines reconstruction loss and perceptual loss to generate more realistic and visually appealing repair results.

Specifically, WSSL leverages different pre-training tasks (such as rotation prediction, saturation classification, etc.) and optimizes the feature learning process by adjusting the weights between different tasks. This weighted mechanism allows the model to allocate different learning resources based on the importance of specific tasks. Experimental results show that WSSL achieves significant improvements in image inpainting quality compared to previous methods.