Abstract:In recent years, there has been a growing demand for the colorization of remote sensing images due to their inherent limitations caused by remote sensors, such as hazy or noisy atmospheric conditions. These factors result in the captured images needing to be clarified. Compared to ordinary images, remote sensing images present unique challenges in color recovery due to their imbalanced spatial distribution of objects. In this article, we propose a novel bidirectional layout-semantic-pixel joint decoupling and embedding network (BDEnet) following the idea of human painting to generate highly saturated color images with strong spatial consistency and object salience. The proposed BDEnet model emulates the process of human painting through a step-by-step approach. It begins by determining the overall tone of a large macroscopic region and progressively refining the local color based on this initial assessment. Specifically, BDEnet incorporates finer-grained semantics and pixel color information into a colored layout that represents a wide range of continuous areas, thereby accomplishing the colorization task. The BDEnet model operates at three scales, namely the layout (macro), semantic (medium), and pixel (micro) scales. It comprises three key modules: the multiscale feature decoupling (MFD) module, the layout-semantic-pixel multigranularity learning (MGL) module, and the semantic-pixel embedding (SPE) module. MFD module effectively reduces redundant noise from the semantic and layout scales by employing scale decoupling. This process ensures the extraction of efficient features essential for MGL. In the MGL module, three branches with different scales are employed to achieve layout division, semantic segmentation, and pixel coloring. To address the issue of insufficient category label guidance in layouts, we propose a novel approach called similar semantic merging (SSM) using a weakly supervised scheme to accomplish layout division. Finally, the SPE module incorporates stable semantic and pixel information into the layout features. This integration results in the generation of color images that exhibit strong spatial consistency, emphasize object salience, and possess high color saturation.

Bidirectional Layout-Semantic-Pixel Joint Decoupling and Embedding Network for Remote Sensing Colorization

Remote Sensing Colorization Based on Bidirectional Macro-Micro Adaptive Enhancement Network

Lightweight Deep Exemplar Colorization Via Semantic Attention-Guided Laplacian Pyramid

SCSNet: an Efficient Paradigm for Learning Simultaneously Image Colorization and Super-resolution

Imbalance Satellite Image Colorization with Semantic Salience Priors

Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization

Global Context Dependencies Aware Network for Efficient Semantic Segmentation of Fine-Resolution Remoted Sensing Images

DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders

DEANet: Dual Encoder with Attention Network for Semantic Segmentation of Remote Sensing Imagery

Learn to Be Clear and Colorful: An End-to-End Network for Panchromatic Image Enhancement

Scale-Semantic Joint Decoupling Network for Image-text Retrieval in Remote Sensing

High-Order Semantic Decoupling Network for Remote Sensing Image Semantic Segmentation

Deep Exemplar-based Colorization

SECBNet: Semantic Segmentation Enhanced Color Balance Network for Optical Satellite Images

Remote Sensing Image Colorization Based on Joint Stream Deep Convolutional Generative Adversarial Networks

Semantic Category Balance-Aware Involved Anti-Interference Network for Remote Sensing Semantic Segmentation

Hierarchical Self-Attention Embedded Neural Network With Dense Connection for Remote-Sensing Image Semantic Segmentation

Boosting Semantic Segmentation of Remote Sensing Images by Introducing Edge Extraction Network and Spectral Indices

Remote sensing image dehazing based on data blending and Laplace network

Efficient Parallel Multi-Scale Detail and Semantic Encoding Network for Lightweight Semantic Segmentation

Boundary-enhanced dual-stream network for semantic segmentation of high-resolution remote sensing images