Abstract:Blind image decomposition aims to decompose all components present in an image, typically used to restore a multi-degraded input image. While fully recovering the clean image is appealing, in some scenarios, users might want to retain certain degradations, such as watermarks, for copyright protection. To address this need, we add controllability to the blind image decomposition process, allowing users to enter which types of degradation to remove or retain. We design an architecture named controllable blind image decomposition network. Inserted in the middle of U-Net structure, our method first decomposes the input feature maps and then recombines them according to user instructions. Advantageously, this functionality is implemented at minimal computational cost: decomposition and recombination are all parameter-free. Experimentally, our system excels in blind image decomposition tasks and can outputs partially or fully restored images that well reflect user intentions. Furthermore, we evaluate and configure different options for the network structure and loss functions. This, combined with the proposed decomposition-and-recombination method, yields an efficient and competitive system for blind image decomposition, compared with current state-of-the-art methods.
Computer Vision and Pattern Recognition,Machine Learning,Image and Video Processing
What problem does this paper attempt to address?
The paper primarily addresses the problem of achieving controllable Blind Image Decomposition (BID) in image processing. Specifically, its goal is to selectively remove or retain certain types of degradations in images containing multiple degradation factors based on user-specific requirements.
The proposed method includes the following key points:
1. **Controllability**: Achieves controllability by integrating user prompts, allowing users to specify which degradation components need to be removed, such as rain, fog, watermarks, etc.
2. **Multi-Degradation Removal Dataset**: Constructs a dataset containing various degradation components, covering weather-related degradations (e.g., rain streaks, snow, haze), illumination-related degradations (e.g., glare, reflection, shadow), and occlusion-related degradations (e.g., fences, watermarks).
3. **Network Architecture**: Proposes the Controllable Blind Image Decomposition Network (CBDNet), an efficient structure utilizing a multi-scale, layered encoder-decoder backbone, combined with decomposition blocks, controllability blocks, and reorganization blocks to effectively follow user intentions.
- **Decomposition Block**: Segments the input feature map into multiple sub-feature maps, each corresponding to a type of degradation.
- **Controllability Block**: Includes a source classifier and a prompt converter, used to predict the degradation components present in the input image and convert user instructions into a form understandable by the model.
- **Reorganization Block**: Selectively reorganizes the feature maps based on user instructions to generate the final restored image.
The experimental section demonstrates the superior performance of the proposed method on various tasks, including multi-weather removal, real-world adverse weather removal, and multi-degradation removal tasks. Through quantitative and qualitative results comparisons, it is confirmed that the method not only excels in image restoration but also performs well in controllable blind image decomposition tasks, effectively removing or retaining specific degradation components based on different user instructions.