Abstract:The creation of manufacturable and editable 3D shapes through Computer-Aided Design (CAD) remains a highly manual and time-consuming task, hampered by the complex topology of boundary representations of 3D solids and unintuitive design tools. This paper introduces GenCAD, a generative model that employs autoregressive transformers and latent diffusion models to transform image inputs into parametric CAD command sequences, resulting in editable 3D shape representations. GenCAD integrates an autoregressive transformer-based architecture with a contrastive learning framework, enhancing the generation of CAD programs from input images and providing a representation learning framework for multiple data modalities relevant to engineering designs. Extensive evaluations demonstrate that GenCAD significantly outperforms existing state-of-the-art methods in terms of the precision and modifiability of generated 3D shapes. Notably, GenCAD shows a marked improvement in the accuracy of 3D shape generation for long sequences, supporting its application in complex design tasks. Additionally, the contrastive embedding feature of GenCAD facilitates the retrieval of CAD models using image queries from databases which is a critical challenge within the CAD community. While most work in the 3D shape generation literature focuses on representations like meshes, voxels, or point clouds, practical engineering applications demand modifiability and the ability for multi-modal conditional generation. Our results provide a significant step forward in this direction, highlighting the potential of generative models to expedite the entire design-to-production pipeline and seamlessly integrate different design modalities.

Fusion 360 Gallery: A Dataset and Environment for Programmatic CAD Construction from Human Design Sequences

Fusion 360 Gallery: A Dataset and Environment for Programmatic CAD Reconstruction

SECAD-Net: Self-Supervised CAD Reconstruction by Learning Sketch-Extrude Operations

An Example-Based Approach to 3D Man-Made Object Reconstruction from Line Drawings

A parametric and feature-based CAD dataset to support human-computer interaction for advanced 3D shape learning

TransCAD: A Hierarchical Transformer for CAD Sequence Inference from Point Clouds

Point2CAD: Reverse Engineering CAD Models from 3D Point Clouds

GenCAD: Image-Conditioned Computer-Aided Design Generation with Transformer-Based Contrastive Representation and Diffusion Priors

Brep2Seq: A dataset and hierarchical deep learning network for reconstruction and generation of computer-aided design models

CLS-CAD: Synthesizing CAD Assemblies in Fusion 360

Embedding Visual Cognition in 3D Reconstruction from Multi-View Engineering Drawings

GenScan: A Generative Method for Populating Parametric 3D Scan Datasets

DesignFusion: Integrating Generative Models for Conceptual Design Enrichment

Seamless Integration of Analysis and Design: Automatic CAD Reconstruction of Post-Analysis Geometries

Development of a Pilot Manufacturing Cyberinfrastructure With an Information Rich Mechanical CAD 3D Model Repository

Sketch2CAD: Sequential CAD Modeling by Sketching in Context

SketchGraphs: A Large-Scale Dataset for Modeling Relational Geometry in Computer-Aided Design

P2CADNet: An End-to-End Reconstruction Network for Parametric 3D CAD Model from Point Clouds

CADOps-Net: Jointly Learning CAD Operation Types and Steps from Boundary-Representations

CAD-Estate: Large-scale CAD Model Annotation in RGB Videos

Photo2CAD: Automated 3D solid reconstruction from 2D drawings using OpenCV