Abstract:Deep generative models such as Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), Diffusion Models, and Transformers, have shown great promise in a variety of applications, including image and speech synthesis, natural language processing, and drug discovery. However, when applied to engineering design problems, evaluating the performance of these models can be challenging, as traditional statistical metrics based on likelihood may not fully capture the requirements of engineering applications. This paper doubles as a review and practical guide to evaluation metrics for deep generative models (DGMs) in engineering design. We first summarize the well-accepted `classic' evaluation metrics for deep generative models grounded in machine learning theory. Using case studies, we then highlight why these metrics seldom translate well to design problems but see frequent use due to the lack of established alternatives. Next, we curate a set of design-specific metrics which have been proposed across different research communities and can be used for evaluating deep generative models. These metrics focus on unique requirements in design and engineering, such as constraint satisfaction, functional performance, novelty, and conditioning. Throughout our discussion, we apply the metrics to models trained on simple-to-visualize 2-dimensional example problems. Finally, we evaluate four deep generative models on a bicycle frame design problem and structural topology generation problem. In particular, we showcase the use of proposed metrics to quantify performance target achievement, design novelty, and geometric constraints. We publicly release the code for the datasets, models, and metrics used throughout the paper at <a class="link-external link-https" href="https://decode.mit.edu/projects/metrics/" rel="external noopener nofollow">this https URL</a>.

A Review of Generative Models for 3D Vehicle Wheel Generation and Synthesis

A Deep Learning-Based CAE Approach for Simulating 3D Vehicle Wheels Under Real-World Conditions

AI-assisted design of lightweight and strong 3D-printed wheels for electric vehicles

Integrating deep learning into CAD/CAE system: generative design and evaluation of 3D conceptual wheel

Deep Generative Models in Engineering Design: A Review

Integrating Deep Learning into CAD/CAE System: Case Study on Road Wheel Design Automation

3D Solid Model Generation Method Based on a Generative Adversarial Network

Deep Generative Model for Mechanical System Configuration Design

A Generative Neural Network Approach for 3D Multi-Criteria Design Generation and Optimization of an Engine Mount for an Unmanned Air Vehicle

Progress and recent trends in generative design

3D Design Using Generative Adversarial Networks and Physics-Based Validation

Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Design

VehicleSDF: A 3D generative model for constrained engineering design via surrogate modeling

Generative AI Models for Different Steps in Architectural Design: A Literature Review

Exploring the Potentials and Challenges of Deep Generative Models in Product Design Conception

Generative Models and Connected and Automated Vehicles: A Survey in Exploring the Intersection of Transportation and AI

Generative AI for Architectural Design: A Literature Review

Deep Generative Models on 3D Representations: A Survey

Towards Goal, Feasibility, and Diversity-Oriented Deep Generative Models in Design

A Review: The Application of Generative Adversarial Network for Mechanical Fault Diagnosis

Inspired by AI? A Novel Generative AI System To Assist Conceptual Automotive Design