Abstract:Self-driving industries usually employ professional artists to build exquisite 3D cars. However, it is expensive to craft large-scale digital assets. Since there are already numerous datasets available that contain a vast number of images of cars, we focus on reconstructing high-quality 3D car models from these datasets. However, these datasets only contain one side of cars in the forward-moving scene. We try to use the existing generative models to provide more supervision information, but they struggle to generalize well in cars since they are trained on synthetic datasets not car-specific. In addition, The reconstructed 3D car texture misaligns due to a large error in camera pose estimation when dealing with in-the-wild images. These restrictions make it challenging for previous methods to reconstruct complete 3D cars. To address these problems, we propose a novel method, named DreamCar, which can reconstruct high-quality 3D cars given a few images even a single image. To generalize the generative model, we collect a car dataset, named Car360, with over 5,600 vehicles. With this dataset, we make the generative model more robust to cars. We use this generative prior specific to the car to guide its reconstruction via Score Distillation Sampling. To further complement the supervision information, we utilize the geometric and appearance symmetry of cars. Finally, we propose a pose optimization method that rectifies poses to tackle texture misalignment. Extensive experiments demonstrate that our method significantly outperforms existing methods in reconstructing high-quality 3D cars. \href{<a class="link-external link-https" href="https://xiaobiaodu.github.io/dreamcar-project/" rel="external noopener nofollow">this https URL</a>}{Our code is available.}

3D car shape reconstruction from a contour sketch using GAN and lazy learning

Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs

DreamCar: Leveraging Car-specific Prior for in-the-wild 3D Car Reconstruction

Progressive Learning of 3D Reconstruction Network from 2D GAN Data

GINA-3D: Learning to Generate Implicit Neural Assets in the Wild

New Fashion: Personalized 3D Design with a Single Sketch Input

3D-CariGAN: An End-to-End Solution to 3D Caricature Generation from Face Photos

Semi-Supervised Learning in Reconstructed Manifold Space for 3d Caricature Generation

Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Data

3D-Mask-GAN:Unsupervised Single-View 3D Object Reconstruction

Shape Inpainting using 3D Generative Adversarial Network and Recurrent Convolutional Networks

Learning single-image 3D reconstruction by generative modelling of shape, pose and shading

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Sketch-A-Shape: Zero-Shot Sketch-to-3D Shape Generation

Lift3D: Synthesize 3D Training Data by Lifting 2D GAN to 3D Generative Radiance Field

Data-Driven Car Drag Prediction With Depth and Normal Renderings

RGM: Reconstructing High-fidelity 3D Car Assets with Relightable 3D-GS Generative Model from a Single Image

Physically‐Based Lighting for 3D Generative Models of Cars

3D-Carigan: an End-to-End Solution to 3D Caricature Generation from Normal Face Photos

3D visualization model construction based on generative adversarial networks

Sketch Your Own GAN