TransCAD: A Hierarchical Transformer for CAD Sequence Inference from Point Clouds

Elona Dupont,Kseniya Cherenkova,Dimitrios Mallis,Gleb Gusev,Anis Kacem,Djamila Aouada
2024-07-18
Abstract:3D reverse engineering, in which a CAD model is inferred given a 3D scan of a physical object, is a research direction that offers many promising practical applications. This paper proposes TransCAD, an end-to-end transformer-based architecture that predicts the CAD sequence from a point cloud. TransCAD leverages the structure of CAD sequences by using a hierarchical learning strategy. A loop refiner is also introduced to regress sketch primitive parameters. Rigorous experimentation on the DeepCAD and Fusion360 datasets show that TransCAD achieves state-of-the-art results. The result analysis is supported with a proposed metric for CAD sequence, the mean Average Precision of CAD Sequence, that addresses the limitations of existing metrics.
Computer Vision and Pattern Recognition,Artificial Intelligence
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to automatically infer the corresponding Computer - Aided Design (CAD) model sequence given a 3D scanned point cloud. Specifically, the paper proposes an end - to - end trainable hierarchical Transformer architecture named TransCAD, aiming to predict CAD sequences from point clouds. This problem is of great significance in the field of reverse engineering because it allows for the automatic generation of CAD models from 3D scans of physical objects, thereby enabling rapid replication and modification of physical objects. ### Core Problems of the Paper - **3D Reverse Engineering**: How to automatically infer CAD models from 3D scans of physical objects. - **CAD Sequence Prediction**: How to predict CAD operation sequences, including 2D sketches and 3D operations (such as extrusion, rotation, etc.), from point cloud data. ### Solutions - **TransCAD Model**: A hierarchical network based on Transformer that can directly predict CAD sequences from point clouds. - **Hierarchical Learning Strategy**: The model adopts a two - layer decoding process. First, it decodes high - level CAD sequence embeddings, and then the secondary decoder processes loop parameters and CAD operations respectively. - **Loop Refiner**: A loop refiner is introduced to regress sketch primitive parameters, further improving the accuracy of the model. ### Main Contributions 1. **Proposing a New Hierarchical Architecture**: TransCAD is a single - stage, end - to - end trainable model that can compactly represent CAD sequences and support step - by - step coordinate refinement. 2. **Improving the Evaluation Framework**: The limitations of the existing evaluation framework are identified, and a new evaluation metric - Average Precision of CAD Sequences (APCS) - is proposed to evaluate model performance more accurately. 3. **Robustness in Practical Applications**: The model performs excellently when dealing with noisy and irregular point cloud data and is suitable for real - world applications. ### Technical Details - **Point Cloud Encoder**: Use a standard point cloud encoder (such as PointNet++) to encode point clouds into point features. - **Loop - Extrusion Decoder**: Learn high - level loop and extrusion embeddings through multi - head Transformer blocks. - **Loop and Extrusion Parameterization**: Use loop decoder and extrusion decoder to predict quantized loop and extrusion parameters respectively. - **Loop Refiner**: Correct the predicted quantized loop parameters through unquantized ground truth values to reduce quantization error. ### Evaluation Methods - **Limitations of Existing Evaluation Methods**: - **Command Type Accuracy (ACC cmd)**: Over - prediction cases are ignored. - **Parameter Accuracy (ACC param)**: Evaluation is only carried out on a subset of accurately identified commands, resulting in complex result interpretation. - **Chamfer Distance (CD)**: Although it can evaluate shape similarity, it cannot reflect the accuracy of CAD sequences. - **Newly Proposed Evaluation Metrics**: - **Average Precision of CAD Sequences (APCS)**: Combine the accuracy of operation type and parameter prediction to ensure that both over - predicted and under - predicted sequences are equally penalized. - **Median Chamfer Distance**: Evaluated on 4096 points to reduce measurement uncertainty. ### Experimental Results - **Quantitative Results**: TransCAD achieves state - of - the - art performance on multiple datasets, especially when dealing with noisy and irregular point cloud data. - **Qualitative Results**: Several qualitative results are presented, demonstrating the effectiveness of the model in generating high - quality CAD models. In conclusion, through proposing the TransCAD model and an improved evaluation framework, this paper effectively solves the problem of automatically inferring CAD models from 3D scanned point clouds, providing a new solution for the field of reverse engineering.