WavShadow: Wavelet Based Shadow Segmentation and Removal

Shreyans Jain,Aadya Arora,Viraj Vekaria,Karan Gandhi

2024-11-09

Abstract:Shadow removal and segmentation remain challenging tasks in computer vision, particularly in complex real-world scenarios. This study presents a novel approach that enhances the ShadowFormer model by incorporating Masked Autoencoder (MAE) priors and Fast Fourier Convolution (FFC) blocks, leading to significantly faster convergence and improved performance. We introduce key innovations: (1) integration of MAE priors trained on Places2 dataset for better context understanding, (2) adoption of Haar wavelet features for enhanced edge detection and multi-scale analysis, and (3) implementation of a modified SAM Adapter for robust shadow segmentation. Extensive experiments on the challenging DESOBA dataset demonstrate that our approach achieves state-of-the-art results, with notable improvements in both convergence speed and shadow removal quality.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenges of shadow segmentation and removal in the field of computer vision, especially in complex real - world scenes. Specifically:

1. **Difficulties in shadow segmentation and removal**: In complex real - world environments, there are complex interactions between shadows and objects, which makes accurate shadow segmentation and removal very difficult.

2. **Limitations of existing methods**:
- Traditional methods rely on physical models and heuristic rules based on chromaticity, intensity, and texture, and these methods have limited effectiveness when dealing with complex scenes.
- Although machine - learning methods have improved by combining hand - designed features (such as texture, color, edges) with classifiers (such as SVM and decision trees), they are still insufficient in comprehensively characterizing shadows.
- Deep - learning methods have significantly improved performance through automatic feature extraction, but they still have problems such as slow convergence speed and poor performance when dealing with complex scenes.

3. **The need to improve performance**: In order to achieve faster convergence speed and higher shadow - removal quality in complex real - world scenes, new techniques and methods need to be introduced to enhance the performance of existing models.

For this purpose, the paper proposes a novel method to improve shadow segmentation and removal by integrating the following key techniques:
- **Masked Autoencoder (MAE) prior**: Use the MAE trained on the Places2 dataset to provide better context understanding.
- **Haar wavelet features**: Used to enhance edge detection and multi - scale analysis and improve the accuracy of shadow segmentation.
- **Improved SAM Adapter**: Improve the effect of shadow segmentation by generating more robust shadow masks.
- **Fast Fourier Convolution (FFC) block**: Used to better capture global context information and accelerate model convergence.

Through these innovations, the paper aims to significantly improve the performance of shadow segmentation and removal, especially in complex real - world scenes. Experimental results show that this method has achieved state - of - the - art results on the DESOBA dataset, showing faster convergence speed and higher shadow - removal quality.