Three stages of 3D virtual try-on network with appearance flow and shape field
Ziyi Chen,Feng Yu,Minghua Jiang,Hua Wang,Ailing Hua,Tao Peng,Xinrong Hu,Ping Zhu
DOI: https://doi.org/10.1007/s00371-023-02946-3
2023-07-26
Abstract:The virtual try-on technology can satisfy the demands for online shopping and help consumers experience online clothes through image generation technology. Compared with image-based try-on, the 3D virtual try-on methods can realize the multi-perspective of try-on simulation and get the attention of many researchers. The current 3D virtual try-on methods mainly base on the thin-plate spline method and depth information-based 3D reconstruction, which increases the costs of implementing 3D virtual try-on and the results lack of clothing details such as folds and patterns. To solve those problems, we propose a novel 3D virtual try-on network based on appearance flow and shape field called AFSF-3DVTON. Specifically, this network consists of three modules. First, the appearance flow warping module generates the desired warped clothes according to the appearance flow of the original clothes. Then, the flat try-on module facilitates geometric matching between the warped clothes and reference person images and synthesizes 2D try-on results. Third, to increase the image's details to 3D try-on synthesis, the shape field-based reconstruction is adopted, which extracts shape features of 2D try-on results to improve the quality of 3D try-on reconstruction. We evaluate the proposed method on the VITON and MPV3D datasets, and several state-of-the-art virtual try-on algorithms are used as comparisons. The qualitative analyses verify the superiority of the proposed method, and the evaluation indexes, including Abs., Sq., and RMSE, demonstrate the outperformance of the proposed network.
computer science, software engineering