Abstract:Human hair segmentation is essential for face recognition and for achieving natural transformation of style transfer. However, it remains a challenging task due to the diverse appearances and complex patterns of hair in image. In this study, we propose a novel method utilizing diffusion-based generative models, which have been extensively researched in recent times, to effectively capture and to finely segment human hair. In diffusion-based models, an internal visual representation during the denoising process contains pixel-level rich information. Inspired by this aspect, we introduce diffusion-based models for segmenting fine-grained human hair. Specifically, we extract the representation from the diffusion-based models, which contains pixel-level semantic information, and then train a segmentation network using it. Particularly, to more finely segment human hair, our approach employs the representation from a text-to-image diffusion model, conditioned on text information, to extract more relevant information for human hair, thereby predicting detailed hair masks. To validate our method, we conducted experiments on three distinct hair-related datasets with unique characteristics: Figaro-1k, CelebAMask-HQ, and Face Synthetics. The experimental results show the improved performance of our proposed method across all three datasets, outperforming existing methods in terms of mIoU (mean intersection over union), accuracy, precision, and F1-score. This is particularly evident in its ability to accurately capture and finely segment human hair from background and non-hair elements. This demonstrates the effectiveness of our method in accurately and finely segmenting human hair with complex characteristics. Our research contributes not only to the fine-grained segmentation of human hair but also to the application of generative models in semantic segmentation tasks. We hope that the proposed method will be applied for detailed semantic segmentation in various fields in the future.

Isomorphic Manifold Inference for Hair Segmentation

Data-driven Hair Segmentation with Isomorphic Manifold Inference

A Novel Coarse-to-fine Hair Segmentation Method.

Hair-GAN: Recovering 3D Hair Structure from a Single Image Using Generative Adversarial Networks

CT2Hair: High-Fidelity 3D Hair Modeling Using Computed Tomography.

A Novel Two-Tier Bayesian Based Method for Hair Segmentation

Struct2Hair: A Hair Shape Descriptor for Hairstyle Modeling.

A Compositional Exemplar-Based Model For Hair Segmentation

Automatic Hair Segmentation in Complex Background.

Unsupervised Multi-Class Co-Segmentation via Joint-Cut Over $L_{1}$ -Manifold Hyper-Graph of Discriminative Image Regions

NeuralHDHair: Automatic High-fidelity Hair Modeling from a Single Image Using Implicit Neural Representations

MonoHair: High-Fidelity Hair Modeling from a Monocular Video

Manifold Embedding and Semantic Segmentation for Intraoperative Guidance with Hyperspectral Brain Imaging

HairStep: Transfer Synthetic to Real Using Strand and Depth Maps for Single-View 3D Hair Modeling

Towards Unified 3D Hair Reconstruction from Single-View Portraits

Hair Style Retrieval by Semantic Mapping on Informative Patches

Fine-Grained Human Hair Segmentation Using a Text-to-Image Diffusion Model

Hair Segmentation on Time-of-Flight RGBD Images

Manifold-manifold distance and its application to face recognition with image sets.

Manifold-Manifold Distance with application to face recognition based on image set

Bidirectional-isomorphic Manifold Learning at Image Semantic Understanding & Representation.