Virtual reconstruction of 3D articulated human shapes applied to garment try-on in a virtual fitting room

Chenxi Li,Fernand Cohen
DOI: https://doi.org/10.1007/s11042-021-11398-7
IF: 2.577
2022-02-17
Multimedia Tools and Applications
Abstract:We introduce an efficient and practical integrated system for human body model personalization with articulation. Starting with a 3D personalized model of the individual in a standard pose, the model is updated to accommodate for changes in articulations captured in a video clip with Ndocumentclass[12pt]{minimal}usepackage{amsmath}usepackage{wasysym}usepackage{amsfonts}usepackage{amssymb}usepackage{amsbsy}usepackage{mathrsfs}usepackage{upgreek}setlength{oddsidemargin}{-69pt}egin{document}$$N$$end{document} frames. The personalized model is segmented into different parts using anthropometric control points on the silhouette boundary of the frontal projection of the 3D model. These are endpoints of segments in 2D corresponding to projections of regions of independently moving parts in 3D. These points can either be manually selected or predicted using a pre-trained convolutional neural network (NN) point model. Model evolution consists of finding a set of 3D transformations that are independently applied to parts on 3D model so that the projections of the 3D model ‘match’ those observed in the video sequence at corresponding frames. This is done by minimizing the error between the frontally projected body region points and the target region points in the image for each independent moving part. The average vertex error of our articulation recovery method yields sub-resolution recovery errors ((about 4.77 mm compared to 17.82 mm—the resolution cell of the body model). This is quite an improvement over the SMPL NN approach using the same error metric that yields 10 times the resolution cell. The virtually reconstructed articulated 3D model is fitted with a 3D garment model for the creation of virtual fitting room that allows an individual to virtually access how well the garment fits.
computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering
What problem does this paper attempt to address?