Abstract:The colorization of grayscale images is a challenging task in image processing. Recently, deep learning has shown remarkable performance in image colorization. However, the detail loss and color distortion are still serious problem for most existing methods, and some useful features may be lost in the processes of various convolutional layers because of the vanishing gradient problem. Therefore, there is still a considerable space to reach the roof of image colorization. In this work, we propose a deep convolutional auto-encoder with special multi-skip connections for image colorization in YUV color space, and the specific contributions or designs of this work are shown as the following five points. First, a given gray image is used as the Y channel to input a deep learning model to predict U and V channel. Second, the adopted encoder-decoder consists of a main path and two branch paths, and the branch path has two skip connection ways that include one shortcut in each three layers and one shortcut in each six layers. Third, the convolutional kernel size is set as 2*2 that is a special consideration in the path of one shortcut in each six layers. Fourth, a composite loss function is proposed based on the mean square error and gradient that is defined to calculate the errors between the ground truth and the predicted result. Finally, we also discuss the reasonable network parameters, such as the way of shortcut connection, the convolutional kernel size of shortcut connection, and loss function parameters. Experiments on different image datasets show that the proposed image colorization model is effective, and the scores of the PNSR, RMSE, SSIM, and Pearson correlation coefficient are, respectively, to 27.0595, 0.1311, 0.561, and 0.9771.

Image Colorization with Dense Feature Fusion

Image Colorization using U-Net with Skip Connections and Fusion Layer on Landscape Images

Color-UNet++: A resolution for colorization of grayscale images using improved UNet++

U-Net-Based Multispectral Image Generation from an RGB Image

Image colorization using deep convolutional auto-encoder with multi-skip connections

ColorUNet: A convolutional classification approach to colorization

Attention-Based Grayscale Image Colorization

Improving reference-based image colorization for line arts via feature aggregation and contrastive learning

Lightweight Deep Exemplar Colorization Via Semantic Attention-Guided Laplacian Pyramid

Learning a Deep Convolutional Network for Colorization in Monochrome-Color Dual-Lens System.

Colorization of gray scale images based on convolutional block attention and Pix2Pix network

Automatic Colorization Using Fully Convolutional Networks

Semantic-driven Colorization

Image Colorization Algorithm Based on Deep Learning

Fast Image Colorization Based on Local and Global Consistency

Night Colorize: Fully Convolutional Colorization Network for Low-Light Images

Colorization Using Neural Network Ensemble.

Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization

Remote Sensing Image Colorization Based on Joint Stream Deep Convolutional Generative Adversarial Networks

User-Guided Anime Line Art Colorization with Spatially-adaptive Normalization

Incorporating Luminance, Depth and Color Information by a Fusion-based Network for Semantic Segmentation