Abstract:Exemplar-based colorization aims to generate plausible colors for a grayscale image with the guidance of a color reference image. The main challenging problem is finding the correct semantic correspondence between the target image and the reference image. However, the colors of the object and background are often confused in the existing methods. Besides, these methods usually use simple encoder-decoder architectures or pyramid structures to extract features and lack appropriate fusion mechanisms, which results in the loss of high-frequency information or high complexity. To address these problems, this paper proposes a lightweight semantic attention-guided Laplacian pyramid network (SAGLP-Net) for deep exemplar-based colorization, exploiting the inherent multi-scale properties of color representations. They are exploited through a Laplacian pyramid, and semantic information is introduced as high-level guidance to align the object and background information. Specially, a semantic guided non-local attention fusion module is designed to exploit the long-range dependency and fuse the local and global features. Moreover, a Laplacian pyramid fusion module based on criss-cross attention is proposed to fuse high frequency components in the large-scale domain. An unsupervised multi-scale multi-loss training strategy is further introduced for network training, which combines pixel loss, color histogram loss, total variance regularisation, and adversarial loss. Experimental results demonstrate that our colorization method achieves better subjective and objective performance with lower complexity than the state-of-the-art methods.

Exemplar-based video colorization with long-term spatiotemporal dependency

Deep Exemplar-Based Color Transfer for 3D Model

STARNet: Low-light Video Enhancement Using Spatio-Temporal Consistency Aggregation

ColorMNet: A Memory-based Deep Spatial-Temporal Feature Propagation Network for Video Colorization

Temporal Consistent Automatic Video Colorization via Semantic Correspondence

Temporally consistent video colorization with deep feature propagation and self-regularization learning

BiSTNet: Semantic Image Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-based Video Colorization

Video Colorization Based on Variational Autoencoder

Reference-based Video Colorization with AB Chrominance Point and Temporal Propagation

Hierarchical Color Fusion Network (HCFN): Enhancing Exemplar-Based Video Colorization

L-C4: Language-Based Video Colorization for Creative and Consistent Color

Deep Exemplar-based Colorization

Lightweight Deep Exemplar Colorization Via Semantic Attention-Guided Laplacian Pyramid

Reference-Based Video Colorization with Spatiotemporal Correspondence

Video Demoiréing with Deep Temporal Color Embedding and Video-Image Invertible Consistency

SCSNet: an Efficient Paradigm for Learning Simultaneously Image Colorization and Super-resolution

CPNet: Continuity Preservation Network for infrared video colorization

Video Colorization Using Parallel Optimization in Feature Space

Stylization-Based Architecture for Fast Deep Exemplar Colorization

Video Colorization: A Survey

Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization