MUNCH: Modelling Unique 'N Controllable Heads

Debayan Deb,Suvidha Tripathi,Pranit Puri

2023-10-04

Abstract:The automated generation of 3D human heads has been an intriguing and challenging task for computer vision researchers. Prevailing methods synthesize realistic avatars but with limited control over the diversity and quality of rendered outputs and suffer from limited correlation between shape and texture of the character. We propose a method that offers quality, diversity, control, and realism along with explainable network design, all desirable features to game-design artists in the domain. First, our proposed Geometry Generator identifies disentangled latent directions and generate novel and diverse samples. A Render Map Generator then learns to synthesize multiply high-fidelty physically-based render maps including Albedo, Glossiness, Specular, and Normals. For artists preferring fine-grained control over the output, we introduce a novel Color Transformer Model that allows semantic color control over generated maps. We also introduce quantifiable metrics called Uniqueness and Novelty and a combined metric to test the overall performance of our model. Demo for both shapes and textures can be found: https://munch-seven.vercel.app/. We will release our model along with the synthetic dataset.

Computer Vision and Pattern Recognition,Artificial Intelligence,Graphics,Machine Learning

What problem does this paper attempt to address?

The main focus of this paper is to address the lack of control and diversity in the automated process of 3D human head modeling. While current methods are able to generate realistic portraits, they fall short in terms of diversity and quality control of the generated results, as well as the limited correlation between shape and texture. The paper proposes an AI-assisted modeling approach called MUNCH, which allows users to control the creation of 3D human head models based on attributes such as age, gender, race, and skin color. This approach consists of three modules: the Geometry Generator, which generates diverse shapes; the Render Maps Generator, which learns to synthesize high-quality physically-based render maps; and the Color Transformer Model, which provides semantic color control over the generated textures. The paper also introduces unique and novel quantitative metrics to evaluate the overall performance of the model and provides demonstrations and synthetic datasets. The goal is to provide game design artists with a 3D asset generation tool that offers high diversity, novelty, correlation, realism, and control.