Dual-Tree Complex Wavelet Transform and Deep CNN-Based Super-Resolution for Video Inpainting with Application to Object Removal and Error Concealment

Gajanan Tudavekar,Sanjay R. Patil,Santosh S. Saraf

DOI: https://doi.org/10.1007/978-3-030-67921-7_12

2021-01-01

Abstract:Video inpainting is a process of filling in the missing regions of a video plausibly. The existing video inpainting methods produce seam in the target region and are also computationally expensive. This is due to the inpainting algorithms’ ineffectiveness to select the best match-patch and the longer execution time is due to applying the inpainting algorithm on the original scale image. Motivated by the shortcomings, we propose a video inpainting framework that performs inpainting on the input course image. This workdown sample scans image to get a low-resolution (LR) image. The wavelet sub-bands of LR images stem from using the dual-tree complex wavelet transform (DTCWT). Then, a Criminisi-based inpainting algorithm and an autoregression procedure process these sub-bands to render the inpainting from the LR images. A high-resolution (HR) output comes as an outcome from one of its low-resolution inpainted versions by applying deep convolutional neural networks (CNN) to single image super-resolution (SISR) and named DTCWT-DCNN. The advantage of the DTCWT-DCNN framework is the gain in computational time and the quality of the inpainted video. The experimental results on the video sequences demonstrate the effectiveness of the proposed method in object removal and error concealment. The results of the DTCWT-DCNN framework are compared with three state-of-the-art inpainting methods.