CTH-Net: CNN-Transformer Hybrid Network for Garment Image Generation from Sketches and Color Points

PAN Dong-hui,JIN Ying-han,SUN Xu,LIU Yu-sheng,ZHANG Dong-liang
DOI: https://doi.org/10.11996/JG.j.2095-302X.2023010120
2023-01-01
Abstract:Drawing garment images is an important part of garment design. To address the problems such as low intelligence and high requirements for users’ drawing skills and imagination, a CNN-Transformer hybrid network(CTH-Net) was proposed to generate garment images from sketches and color points. CTH-Net combined the advantages of convolutional neural networks(CNN) in extracting local information and Transformer in processing long-range dependencies, efficiently fusing the architectures of these two models. The ToPatch and ToFeatureMap modules were also designed to reduce the amount and dimension of data input into Transformer, thus reducing the consumption of computing resources. CTH-Net consisted of three phases: the first drafting phase, which aimed to predict the color distribution of garments and obtain watercolor images without gradients and shadows; the second refinement phase, which refined the watercolor image into a realistic garment image; the third tuning phase, which combined the outputs of the above two phases to further optimize the generation quality. The experimental results show that CTH-Net could generate high-quality garment images by simply inputting sketches and some color points.The proposed network could outperform the existing methods in the realism and accuracy of the generated images.
What problem does this paper attempt to address?