Abstract:The demand for high-quality and diverse 3D content has increased due to its applications in virtual reality, augmented reality, and 3D printing. Converting 2D images to 3D models is a challenging task requiring an understanding of depth, texture, and illumination. This study delves into a novel deep learning-based methodology for generating high-quality and accurate 3D models from 2D images. The proposed approach combines Generative Adversarial Networks (GANs) and Deep Marching Tetrahedra to synthesize complex 3D objects with realistic textures and lighting effects. Additionally, a 2D Texture Generator based on Random Noise and a GAN, as well as a Light Map Generator using Spectral Power Distribution Function and a GAN, are designed to enhance the visual appeal and realism of the generated 3D models. The paper presents a tri-model architecture incorporating a seam-aware landmark detector, which identifies heatmaps to ensure precise mapping of 2D textures onto the 3D model. This feature significantly improves the accuracy and quality of texture application by aligning key points from the 2D images with corresponding areas on the 3D geometry. Furthermore, the model employs point source lighting for light map generation, simulating realistic illumination effects that contribute to the final output's visual richness. The proposed technique is evaluated, showcasing its superiority over existing methods in generating diverse and realistic 3D models. The study highlights the potential applications of the proposed technique in various domains, including computer graphics, virtual reality, architecture, and industrial design. The ability to generate accurate 3D models with diverse variations opens up exciting opportunities for design exploration and visualization. This work contributes transformative solutions for 3D object synthesis, 2D texture generation, and light map simulation, paving the way for advancements in 3D modeling and design.

Procedural 3D Terrain Generation using Generative Adversarial Networks

SDF-based 3D Generative Adversarial Networks for Image and Geometry Generation

3D-Aware Image Synthesis Via Learning Structural and Textural Representations

A step towards procedural terrain generation with GANs

SpatialGAN: Progressive Image Generation Based on Spatial Recursive Adversarial Expansion

Generative 3D Reconstruction of Martian Surfaces Using Monocular Images

Deep Generative Framework for Interactive 3D Terrain Authoring and Manipulation

Prototype Learning Based Realistic 3D Terrain Generation from User Semantics

GAN-Based Content Generation of Maps for Strategy Games

Procedural terrain generation with style transfer

Realistic 3D Object Generation Using Seam Aware Landmark Detectors with Texture and Lighting

Latent-Space Laplacian Pyramids for Adversarial Representation Learning with 3D Point Clouds

A GAN-based Active Terrain Mapping for Collaborative Air-Ground Robotic System.

Subsurface Depths Structure Maps Reconstruction with Generative Adversarial Networks

Reconstructing High-Resolution DEMs from 3D Terrain Features Using Conditional Generative Adversarial Networks

Example-based rapid generation of vegetation on terrain via CNN-based distribution learning

Deep Snow: Synthesizing Remote Sensing Imagery with Generative Adversarial Nets

Conditional Progressive Generative Adversarial Network for satellite image generation

Generating Realistic Geology Conditioned on Physical Measurements with Generative Adversarial Networks

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Parametrization and generation of geological models with generative adversarial networks