Shadow-aware image colorization

DOI: https://doi.org/10.1007/s00371-024-03500-5

IF: 2.835

2024-06-05

The Visual Computer

Abstract:Significant advancements have been made in colorization in recent years, especially with the introduction of deep learning technology. However, challenges remain in accurately colorizing images under certain lighting conditions, such as shadow. Shadows often cause distortions and inaccuracies in object recognition and visual data interpretation, impacting the reliability and effectiveness of colorization techniques. These problems often lead to unsaturated colors in shadowed images and incorrect colorization of shadows as objects. Our research proposes the first shadow-aware image colorization method, addressing two key challenges that previous studies have overlooked: integrating shadow information with general semantic understanding and preserving saturated colors while accurately colorizing shadow areas. To tackle these challenges, we develop a dual-branch shadow-aware colorization network. Additionally, we introduce our shadow-aware block, an innovative mechanism that seamlessly integrates shadow-specific information into the colorization process, distinguishing between shadow and non-shadow areas. This research significantly improves the accuracy and realism of image colorization, particularly in shadow scenarios, thereby enhancing the practical application of colorization in real-world scenarios.

computer science, software engineering

What problem does this paper attempt to address?

This paper attempts to solve the problem of inaccurate processing of shadow areas during image coloring. Specifically, existing image coloring techniques face two main challenges when dealing with shadow areas:

1. **Integration of shadow information and general semantic understanding**: Information in shadow areas needs to be effectively combined with other semantic information in the image to improve the accuracy of coloring.
2. **Maintaining saturation and color richness**: While accurately coloring shadow areas, maintain the color saturation and richness of non - shadow areas.

These problems lead to issues such as color distortion in shadow areas and a decrease in color saturation in non - shadow areas, especially being particularly evident in real - world images with shadows. To solve these problems, the paper proposes a new "shadow - aware image coloring" method. By designing a two - branch network to handle shadow detection and shadow - aware coloring tasks respectively, and introducing an innovative shadow - aware module that can effectively separate and process shadow areas and non - shadow areas, it significantly improves the accuracy and realism of image coloring, especially in shadow scenes.