Abstract:In the monochrome-color dual-lens system, the gray image captured by the monochrome camera has better quality than the color image from the color camera, but does not have color information. To get high-quality color images, it is desired to colorize the gray image with the color image as reference. Related works usually use hand-crafted methods to search for the best-matching pixel in the reference image for each pixel in the input gray image, and copy the color of the best-matching pixel as the result. We propose a novel deep convolution network to solve the colorization problem in an end-to-end way. Based on our observation that, for each pixel in the input image, there usually exist multiple pixels in the reference image that have the correct colors, our method performs weighted average of colors of the candidate pixels in the reference image to utilize more candidate pixels with correct colors. The weight values between pixels in the input image and the reference image are obtained by learning a weight volume using deep feature representations, where an attention operation is proposed to focus on more useful candidate pixels and a 3-D regulation is performed to learn with context information. In addition, to correct wrongly colorized pixels in occlusion regions, we propose a color residue joint learning module to correct the colorization result with the input gray image as guidance. We evaluate our method on the Scene Flow, Cityscapes, Middlebury, and Sintel datasets. Experimental results show that our method largely outperforms the state-of-the-art methods.

Deep Colormap Extraction from Visualizations

Deep Exemplar-Based Color Transfer for 3D Model

ColorNetVis: an Interactive Color Network Analysis System for Exploring the Color Composition of Traditional Chinese Painting

Extracting and Retargeting Color Mappings from Bitmap Images of Visualizations

Adaptive Colormap Optimization Based on Inserting Colors

Deep Exemplar-based Colorization

Deep Koalarization: Image Colorization using CNNs and Inception-ResNet-v2

A Survey of Colormaps in Visualization

Data-driven Colormap Adjustment for Exploring Spatial Variations in Scalar Fields

Adaptive transfer of color from images to maps and visualizations

Real-Time User-Guided Image Colorization with Learned Deep Priors

Data-Driven Colormap Optimization for 2D Scalar Field Visualization.

Evaluating the visualization of what a Deep Neural Network has learned

CMA: An End-to-End System for Reverse Engineering Choropleth Map Images

MultiColor: Image Colorization by Learning from Multiple Color Spaces

Saliency Guided Deep Neural Network for Color Transfer With Light Optimization

DeepEye: Towards Automatic Data Visualization

Panoptic-Depth Color Map for Combination of Depth and Image Segmentation

Colorization of Depth Map via Disentanglement

Learning a Deep Convolutional Network for Colorization in Monochrome-Color Dual-Lens System.

Colorizing the Past: Deep Learning for the Automatic Colorization of Historical Aerial Images