Abstract:In recent years, there has been a growing demand for the colorization of remote sensing images due to their inherent limitations caused by remote sensors, such as hazy or noisy atmospheric conditions. These factors result in the captured images needing to be clarified. Compared to ordinary images, remote sensing images present unique challenges in color recovery due to their imbalanced spatial distribution of objects. In this article, we propose a novel bidirectional layout-semantic-pixel joint decoupling and embedding network (BDEnet) following the idea of human painting to generate highly saturated color images with strong spatial consistency and object salience. The proposed BDEnet model emulates the process of human painting through a step-by-step approach. It begins by determining the overall tone of a large macroscopic region and progressively refining the local color based on this initial assessment. Specifically, BDEnet incorporates finer-grained semantics and pixel color information into a colored layout that represents a wide range of continuous areas, thereby accomplishing the colorization task. The BDEnet model operates at three scales, namely the layout (macro), semantic (medium), and pixel (micro) scales. It comprises three key modules: the multiscale feature decoupling (MFD) module, the layout-semantic-pixel multigranularity learning (MGL) module, and the semantic-pixel embedding (SPE) module. MFD module effectively reduces redundant noise from the semantic and layout scales by employing scale decoupling. This process ensures the extraction of efficient features essential for MGL. In the MGL module, three branches with different scales are employed to achieve layout division, semantic segmentation, and pixel coloring. To address the issue of insufficient category label guidance in layouts, we propose a novel approach called similar semantic merging (SSM) using a weakly supervised scheme to accomplish layout division. Finally, the SPE module incorporates stable semantic and pixel information into the layout features. This integration results in the generation of color images that exhibit strong spatial consistency, emphasize object salience, and possess high color saturation.

Remote Sensing Image Colorization Based on Deep Neural Networks with Multi-Scale Residual Receptive Filed

Deep Neural Network Joint Multi-Scale Attention for Remote Sensing Image Colorization

CSRDNN: an Integrated Scheme for Single Satellite Image Colorization and Super-Resolution Using Deep Neural Networks.

Remote Sensing Scene Image Classification Model Based on Multi-Scale Features and Attention Mechanism

Remote Sensing Image Colorization Based on Multiscale SEnet GAN

A Deep Multitask Convolutional Neural Network for Remote Sensing Image Super-Resolution and Colorization

Image Colorization Algorithm Based on Deep Learning

Remote Sensing Image Colorization Based on Joint Stream Deep Convolutional Generative Adversarial Networks

Remote Sensing Image Colorization Using Symmetrical Multi-Scale DCGAN in YUV Color Space.

SCSNet: an Efficient Paradigm for Learning Simultaneously Image Colorization and Super-resolution

Research on Image Colorization Algorithm Based on Residual Neural Network.

Bidirectional Layout-Semantic-Pixel Joint Decoupling and Embedding Network for Remote Sensing Colorization

Edge-Aware Multi-Scale Progressive Colorization.

Remote Sensing Colorization Based on Bidirectional Macro-Micro Adaptive Enhancement Network

Colorization Using Neural Network Ensemble.

Image colorization using deep convolutional auto-encoder with multi-skip connections

Endoscopic Image Colorization Using Convolutional Neural Network

CASR-Net: A Color-Aware Super-Resolution Network for Panchromatic Image

Color-UNet plus plus : A resolution for colorization of grayscale images using improved UNet plus

Color-UNet++: A resolution for colorization of grayscale images using improved UNet++

Attention-Based Grayscale Image Colorization