3D-PreMise: Can Large Language Models Generate 3D Shapes with Sharp Features and Parametric Control?

Zeqing Yuan,Haoxuan Lan,Qiang Zou,Junbo Zhao

2024-01-12

Abstract:Recent advancements in implicit 3D representations and generative models have markedly propelled the field of 3D object generation forward. However, it remains a significant challenge to accurately model geometries with defined sharp features under parametric controls, which is crucial in fields like industrial design and manufacturing. To bridge this gap, we introduce a framework that employs Large Language Models (LLMs) to generate text-driven 3D shapes, manipulating 3D software via program synthesis. We present 3D-PreMise, a dataset specifically tailored for 3D parametric modeling of industrial shapes, designed to explore state-of-the-art LLMs within our proposed pipeline. Our work reveals effective generation strategies and delves into the self-correction capabilities of LLMs using a visual interface. Our work highlights both the potential and limitations of LLMs in 3D parametric modeling for industrial applications.

Graphics,Artificial Intelligence,Computation and Language

What problem does this paper attempt to address?

The problem discussed in this paper is how to accurately model 3D shapes with sharp features under parameter control, especially in the field of industrial design and manufacturing. Current methods struggle to preserve sharp features in engineering semantics when converting implicit 3D representations into explicit forms. To address this, the researchers propose a framework that utilizes large language models (LLMs) to generate text-driven 3D shapes and manipulate 3D modeling software through program synthesis. They create the 3D-PreMise dataset specifically for 3D parameter modeling of industrial shapes and evaluate the capabilities of advanced LLMs through experiments.

The main contributions of the paper are:
1. Introducing a self-correcting framework for 3D shape generation that utilizes LLMs to control 3D software through code.
2. Constructing a benchmark dataset, 3D-PreMise, for analyzing the performance of cutting-edge LLMs.
3. Exploring effective generation strategies and the self-correcting ability of LLMs through a multi-modal interface.

In their research, the authors find the potential of LLMs in 3D parameter modeling, but also identify challenges such as spatial reasoning, geometric computation, and commonsense reasoning. Through the 3D-PreMise dataset, they evaluate the performance of LLMs in generating industrial objects with accurate dimensional parameters and improve accuracy through iterative self-correction.