Abstract:This paper addresses a new virtual try-on problem of fitting any size of clothes to a reference person in the image domain. While previous image-based virtual try-on methods can produce highly natural try-on images, these methods fit the clothes on the person without considering the relative relationship between the physical sizes of the clothes and the person. Different from these methods, our method achieves size-variable virtual try-on in which the image size of the try-on clothes is changed depending on this relative relationship of the physical sizes. To relieve the difficulty in maintaining the physical size of the closes while synthesizing the high-fidelity image of the whole clothes, our proposed method focuses on the residual between the silhouettes of the clothes in the reference and try-on images. We also develop a size-variable virtual try-on dataset consisting of 1,524 images provided by 26 subjects. Furthermore, we propose an evaluation metric for size-variable virtual-try-on. Quantitative and qualitative experimental results show that our method can achieve size-variable virtual try-on better than general virtual try-on methods.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the deficiencies in existing virtual try - on methods when dealing with the physical size relationship between clothes and the human body. Specifically, although existing image - based virtual try - on methods can generate very natural try - on images, these methods do not consider the relative physical size relationship between clothes and the human body when "putting on" clothes on the human body. Therefore, when users try on clothes of different sizes, the generated images cannot accurately reflect the actual size changes of the clothes. To solve this problem, this paper proposes a new problem - **size - variable virtual try - on**, whose goal is to consider their physical sizes (such as user preferences) when matching the try - on clothes with the human body image, in order to achieve a more realistic try - on effect. ### Main contributions 1. **Size - variable virtual try - on**: A new problem is defined, that is, adjusting the image size of the try - on clothes according to the physical size of the clothes provided by the user. 2. **Size - variable Mask Deformation Network (MDN)**: A new network structure is proposed to adjust the mask according to the physical size of the clothes while maintaining the overall contour of the clothes. The network achieves this by focusing on the residuals between the clothing contours in the reference image and the try - on image. 3. **Size Evaluation Metric (SEM)**: In order to evaluate the effect of size - variable virtual try - on, a new evaluation metric is proposed, which mainly evaluates the size differences in the hem and sleeve areas of the clothes. 4. **Size - variable virtual try - on dataset**: A new dataset is constructed, containing 1,524 images provided by 26 subjects, covering 14 different types of clothes, for training and evaluating the model. ### Method overview - **Mask Deformation Network (MDN)**: This network is divided into two subtasks. One is to adjust the mask according to the physical size of the clothes, and the other is to render the texture into the deformed mask. MDN maintains the overall contour by focusing on the residuals between the clothing contours in the reference image and the try - on image, while adjusting the image size to conform to the physical size. - **Dataset collection**: In order to ensure that the postures of the figures in each pair of images in the dataset are similar, the author proposes a pose - matching method, which selects the most similar frames by calculating the mean - squared error (MSE) between multiple key points. - **Evaluation metric (SEM)**: SEM evaluates the performance of the model by comparing the differences between the generated mask and the real mask in key areas (such as the hem and sleeves). ### Experimental results The experimental results show that the proposed method is superior to the existing methods in the size - variable virtual try - on task. Especially in terms of the size evaluation metric (SEM), this method performs excellently and can more accurately reflect the actual size changes of the clothes. In conclusion, this paper significantly improves the ability of virtual try - on technology in dealing with the size relationship between clothes and the human body by introducing new problems, methods and evaluation metrics.

Size-Variable Virtual Try-On with Physical Clothes Size

Image-Based Virtual Try-on System With Clothing-Size Adjustment

Toward Realistic Virtual Try-on Through Landmark Guided Shape Matching

Clothes Size Prediction from Dressed-Human Silhouettes.

High-Resolution Virtual Try-On Network with Coarse-to-Fine Strategy

Better Fit: Accommodate Variations in Clothing Types for Virtual Try-on

ClothFit: Cloth-Human-Attribute Guided Virtual Try-On Network Using 3D Simulated Dataset

Arbitrary Virtual Try-On Network: Characteristics Preservation and Trade-off between Body and Clothing

Toward Detail-Oriented Image-Based Virtual Try-On with Arbitrary Poses

MM-VTON: A Multi-stage Virtual Try-on Method Using Multiple Image Features.

LC-VTON: Length Controllable Virtual Try-on Network

Image-Based Virtual Try-On: A Survey

Street TryOn: Learning In-the-Wild Virtual Try-On from Unpaired Person Images

Virtual Try-On with Garment Self-Occlusion Conditions

VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization

StyleVTON: A multi-pose virtual try-on with identity and clothing detail preservation

VTNCT: an Image-Based Virtual Try-on Network by Combining Feature with Pixel Transformation

Template-Free Try-on Image Synthesis via Semantic-guided Optimization

FashionOn