Abstract:We introduce a method for assigning photorealistic relightable materials to 3D shapes in an automatic manner. Our method takes as input a photo exemplar of a real object and a 3D object with segmentation, and uses the exemplar to guide the assignment of materials to the parts of the shape, so that the appearance of the resulting shape is as similar as possible to the exemplar. To accomplish this goal, our method combines an image translation neural network with a material assignment neural network. The image translation network translates the color from the exemplar to a projection of the 3D shape and the part segmentation from the projection to the exemplar. Then, the material prediction network assigns materials from a collection of realistic materials to the projected parts, based on the translated images and perceptual similarity of the materials. One key idea of our method is to use the translation network to establish a correspondence between the exemplar and shape projection, which allows us to transfer materials between objects with diverse structures. Another key idea of our method is to use the two pairs of (color, segmentation) images provided by the image translation to guide the material assignment, which enables us to ensure the consistency in the assignment. We demonstrate that our method allows us to assign materials to shapes so that their appearances better resemble the input exemplars, improving the quality of the results over the state-of-the-art method, and allowing us to automatically create thousands of shapes with high-quality photorealistic materials. Code and data for this paper are available at <a class="link-external link-https" href="https://github.com/XiangyuSu611/TMT" rel="external noopener nofollow">this https URL</a>.

Controlling Material Appearance by Examples

ControlMat: A Controlled Generative Approach to Material Capture

AdaCM: Adaptive ColorMLP for Real-Time Universal Photo-realistic Style Transfer

TileGen: Tileable, Controllable Material Generation and Capture

MaterialGAN

Style Fader Generative Adversarial Networks for Style Degree Controllable Artistic Style Transfer

L0 Gradient-Preserving Color Transfer

Guided Fine-Tuning for Large-Scale Material Transfer

Controlling Geometric Abstraction and Texture for Artistic Images

Photo-to-Shape Material Transfer for Diverse Structures

MaterialGAN: Reflectance Capture using a Generative SVBRDF Model

MaterialPicker: Multi-Modal Material Generation with Diffusion Transformers

PhotoMat: A Material Generator Learned from Single Flash Photos

StableMaterials: Enhancing Diversity in Material Generation via Semi-Supervised Learning

Photorealistic Material Editing Through Direct Image Manipulation

Control Method of Perceptual Elements Based on Universal Neural Style Transfer

3-D Material Style Transfer for Reconstructing Unknown Appearance in Complex Natural Materials

Alchemist: Parametric Control of Material Properties with Diffusion Models

DiffMat: Latent diffusion models for image-guided material generation

Material Anything: Generating Materials for Any 3D Object via Diffusion

Adaptive transfer of color from images to maps and visualizations