Abstract:Prototyping complex computer-aided design (CAD) models in modern softwares can be very time-consuming. This is due to the lack of intelligent systems that can quickly generate simpler intermediate parts. We propose Text2CAD, the first AI framework for generating text-to-parametric CAD models using designer-friendly instructions for all skill levels. Furthermore, we introduce a data annotation pipeline for generating text prompts based on natural language instructions for the DeepCAD dataset using Mistral and LLaVA-NeXT. The dataset contains $\sim170$K models and $\sim660$K text annotations, from abstract CAD descriptions (e.g., generate two concentric cylinders) to detailed specifications (e.g., draw two circles with center $(x,y)$ and radius $r_{1}$, $r_{2}$, and extrude along the normal by $d$...). Within the Text2CAD framework, we propose an end-to-end transformer-based auto-regressive network to generate parametric CAD models from input texts. We evaluate the performance of our model through a mixture of metrics, including visual quality, parametric precision, and geometrical accuracy. Our proposed framework shows great potential in AI-aided design applications. Our source code and annotations will be publicly available.

What problem does this paper attempt to address?

The paper attempts to address the issue of the lengthy time required by modern Computer-Aided Design (CAD) software to create complex models. Specifically, existing CAD tools lack intelligent systems capable of quickly generating simple intermediate components. To solve this problem, the paper proposes the Text2CAD framework, an AI-based system that can generate parametric CAD models from designer-friendly text instructions, suitable for users of all skill levels. ### Main Issues: 1. **Time Consumption**: Creating complex CAD models requires a significant amount of time and effort. 2. **Lack of Intelligent Systems**: Existing CAD tools lack intelligent systems that can quickly generate simple intermediate components. 3. **Text to CAD Model Conversion**: There is currently no system capable of generating parametric CAD models from text descriptions. ### Solution: - **Text2CAD Framework**: This is the first AI framework that generates parametric CAD models from text instructions. - **Data Annotation Pipeline**: To generate a large dataset containing text descriptions, the paper introduces a data annotation pipeline that uses Large Language Models (LLMs) and Vision-Language Models (VLMs) to generate text prompts. - **End-to-End Transformer Network**: The paper proposes a Transformer-based autoregressive network that can generate CAD design history from input text. ### Specific Contributions: 1. **Proposing Text2CAD**: As the first AI framework to generate parametric 3D CAD models from text descriptions. 2. **Data Annotation Pipeline**: Utilizing LLMs and VLMs to generate a dataset of text prompts with varying complexity and parameter details. 3. **End-to-End Transformer Architecture**: An end-to-end architecture that generates CAD design history from input text prompts. 4. **Experimental Analysis**: Demonstrating the superior performance of the proposed method in terms of visual quality, parameter accuracy, and geometric precision. Through these innovations, the Text2CAD framework aims to improve the efficiency and usability of CAD design, particularly in the rapid prototyping of complex models.

Text2CAD: Generating Sequential CAD Models from Beginner-to-Expert Level Text Prompts

Text2CAD: Text to 3D CAD Generation via Technical Drawings

CAD Translator: an Effective Drive for Text to 3D Parametric Computer-Aided Design Generative Modeling

Query2CAD: Generating CAD models using natural language queries

GenCAD: Image-Conditioned Computer-Aided Design Generation with Transformer-Based Contrastive Representation and Diffusion Priors

Computer-Aided Design as Language

FlexCAD: Unified and Versatile Controllable CAD Generation with Fine-tuned Large Language Models

Sketch2CAD: Sequential CAD Modeling by Sketching in Context

eCAD-Net: Editable Parametric CAD Models Reconstruction from Dumb B-Rep Models Using Deep Neural Networks

Vitruvion: A Generative Model of Parametric CAD Sketches

QueryCAD: Grounded Question Answering for CAD Models

CAD-Prompted Generative Models: A Pathway to Feasible and Novel Engineering Designs

CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLM

Large Language and Text-to-3D Models for Engineering Design Optimization

CAD-LLM: Large Language Model for CAD Generation

Point2CAD: Reverse Engineering CAD Models from 3D Point Clouds

OpenECAD: An Efficient Visual Language Model for Editable 3D-CAD Design

Text2VP: Generative AI for Visual Programming and Parametric Modeling

Generating CAD Code with Vision-Language Models for 3D Designs

Text2BIM: Generating Building Models Using a Large Language Model-based Multi-Agent Framework

CadVLM: Bridging Language and Vision in the Generation of Parametric CAD Sketches