Detailed Garment Recovery from a Single-View Image

Shan Yang,Tanya Ambert,Zherong Pan,Ke Wang,Licheng Yu,Tamara Berg,Ming C. Lin

DOI: https://doi.org/10.48550/arXiv.1608.01250

2016-09-13

Abstract:Most recent garment capturing techniques rely on acquiring multiple views of clothing, which may not always be readily available, especially in the case of pre-existing photographs from the web. As an alternative, we pro- pose a method that is able to compute a rich and realistic 3D model of a human body and its outfits from a single photograph with little human in- teraction. Our algorithm is not only able to capture the global shape and geometry of the clothing, it can also extract small but important details of cloth, such as occluded wrinkles and folds. Unlike previous methods using full 3D information (i.e. depth, multi-view images, or sampled 3D geom- etry), our approach achieves detailed garment recovery from a single-view image by using statistical, geometric, and physical priors and a combina- tion of parameter estimation, semantic parsing, shape recovery, and physics- based cloth simulation. We demonstrate the effectiveness of our algorithm by re-purposing the reconstructed garments for virtual try-on and garment transfer applications, as well as cloth animation for digital characters.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to recover a detailed clothing model from a single image. Specifically, the author proposes a method that can calculate the 3D models of the human body and the clothing it wears from a single photo with very little human interaction. This method can not only capture the overall shape and geometric structure of the clothing, but also extract the subtle but important details on the clothing, such as hidden folds and wrinkles. Different from the previous methods that rely on complete 3D information (such as depth, multi - view images or sampled 3D geometries), the method in this paper achieves the goal of detailed clothing recovery from a single - view image through the combination of using statistics, geometry and physical priors as well as parameter estimation, semantic parsing, shape recovery and physics - based cloth simulation.

The main contributions of the paper include:
- Proposing an image - guided clothing parameter selection method, which makes it simple and natural to generate virtual clothing with diverse styles and sizes.
- Proposing a joint material - pose optimization framework, which can reconstruct body and clothing models containing material properties from a single image.
- Applying this technology to virtual fitting and character animation, demonstrating its potential in practical applications.

This research solves several limitations existing in the current virtual fitting systems. For example, most systems can only handle fixed virtual character postures, do not consider the influence of different conditions (such as weather changes, posture changes, weight fluctuations, etc.) on cloth materials, and almost all virtual fitting systems assume that users choose predefined virtual characters or obtain accurate body measurement data through 3D scanning. In contrast, the method in this paper can recover a detailed clothing model from a single - view image, allowing users to conduct virtual fitting only with a photo of themselves wearing clothes.