Tongue crack recognition using segmentation based deep learning
Jianjun Yan,Jinxing Cai,Zi Xu,Rui Guo,Wei Zhou,Haixia Yan,Zhaoxia Xu,Yiqin Wang
DOI: https://doi.org/10.1038/s41598-022-27210-x
2023-01-10
Abstract:Tongue cracks refer to fissures with different depth and shapes on the tongue's surface, which can characterize the pathological characteristics of spleen and stomach. Tongue cracks are of great significance to the objective study of tongue diagnosis. However, tongue cracks are small and complex, existing methods are difficult to extract them effectively. In order to achieve more accurate extraction and identification of tongue crack, this paper proposes to apply a deep learning network based on image segmentation (Segmentation-Based Deep-Learning, SBDL) to extract and identify tongue crack. In addition, we have studied the quantitative description of tongue crack features. Firstly, the pre-processed tongue crack samples were amplified by using adding salt and pepper noise, changing the contrast and horizontal mirroring; secondly, the annotation tool Crack-Tongue was used to label tongue crack; thirdly, the tongue crack extraction model was trained by using SBDL; fourthly, the cracks on the tongue surface were detected and located by the segmentation network, and then the output and features of the segmentation network were put into the decision network for the classification of crack tongue images; finally, the tongue crack segmentation and identification results were quantitatively evaluated. The experimental results showed that the tongue crack extraction and recognition results based on SBDL were better than Mask Region-based Convolutional Neural Network (Mask R-CNN), DeeplabV3+, U-Net, UNet++ and Semantic Segmentation with Adversarial Learning (SegAN). This method effectively solved the inaccurate tongue crack extraction caused by the tongue crack's color being close to the surrounding tongue coating's color. This method can achieve better tongue crack extraction and recognition results on a small tongue crack data set and provides a new idea for tongue crack recognition, which is of practical value for tongue diagnosis objectification.