High-Precision Dichotomous Image Segmentation via Probing Diffusion Capacity

Qian Yu,Peng-Tao Jiang,Hao Zhang,Jinwei Chen,Bo Li,Lihe Zhang,Huchuan Lu

2024-10-14

Abstract:In the realm of high-resolution (HR), fine-grained image segmentation, the primary challenge is balancing broad contextual awareness with the precision required for detailed object delineation, capturing intricate details and the finest edges of objects. Diffusion models, trained on vast datasets comprising billions of image-text pairs, such as SD V2.1, have revolutionized text-to-image synthesis by delivering exceptional quality, fine detail resolution, and strong contextual awareness, making them an attractive solution for high-resolution image segmentation. To this end, we propose DiffDIS, a diffusion-driven segmentation model that taps into the potential of the pre-trained U-Net within diffusion models, specifically designed for high-resolution, fine-grained object segmentation. By leveraging the robust generalization capabilities and rich, versatile image representation prior of the SD models, coupled with a task-specific stable one-step denoising approach, we significantly reduce the inference time while preserving high-fidelity, detailed generation. Additionally, we introduce an auxiliary edge generation task to not only enhance the preservation of fine details of the object boundaries, but reconcile the probabilistic nature of diffusion with the deterministic demands of segmentation. With these refined strategies in place, DiffDIS serves as a rapid object mask generation model, specifically optimized for generating detailed binary maps at high resolutions, while demonstrating impressive accuracy and swift processing. Experiments on the DIS5K dataset demonstrate the superiority of DiffDIS, achieving state-of-the-art results through a streamlined inference process. Our code will be made publicly available.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenge of balancing extensive context awareness and the required detail precision in high - resolution, fine - grained image segmentation. Specifically, the paper focuses on how to maintain an understanding of a large - scale background while capturing the fine details and the finest edges of complex objects. Although traditional convolutional neural networks (CNNs) and Transformer - based methods have powerful feature extraction capabilities, they often struggle to simultaneously expand the receptive field and preserve details when processing high - resolution images.

To address this challenge, the paper proposes a diffusion - driven segmentation model named DiffDIS. This model utilizes a pre - trained U - Net (the core component of the diffusion model), combined with a task - specific stable one - step denoising method, which significantly reduces the inference time while maintaining high - fidelity detailed generation. In addition, the paper also introduces an auxiliary edge - generation task to enhance the preservation of object - boundary details and reconcile the stochastic nature of diffusion with the deterministic requirements of the segmentation task.

Through these improvement strategies, DiffDIS can quickly generate detailed binary maps, which are particularly suitable for fine - grained object segmentation of high - resolution images, while demonstrating excellent accuracy and fast processing capabilities. Experimental results show that DiffDIS achieves state - of - the - art performance on the DIS5K dataset and achieves excellent results by simplifying the inference process.