LLM4DESIGN: An Automated Multi-Modal System for Architectural and Environmental Design

Ran Chen,Xueqi Yao,Xuhui Jiang
2024-06-28
Abstract:This study introduces LLM4DESIGN, a highly automated system for generating architectural and environmental design proposals. LLM4DESIGN, relying solely on site conditions and design requirements, employs Multi-Agent systems to foster creativity, Retrieval Augmented Generation (RAG) to ground designs in realism, and Visual Language Models (VLM) to synchronize all information. This system resulting in coherent, multi-illustrated, and multi-textual design schemes. The system meets the dual needs of narrative storytelling and objective drawing presentation in generating architectural and environmental design proposals. Extensive comparative and ablation experiments confirm the innovativeness of LLM4DESIGN's narrative and the grounded applicability of its plans, demonstrating its superior performance in the field of urban renewal design. Lastly, we have created the first cross-modal design scheme dataset covering architecture, landscape, interior, and urban design, providing rich resources for future research.
Human-Computer Interaction,Artificial Intelligence
What problem does this paper attempt to address?
### Problems the Paper Attempts to Solve This paper introduces the LLM4DESIGN system, a highly automated system for generating architectural and environmental design proposals. The system aims to address the following three main challenges: 1. **Generation of Creative Stories**: Design documents need to include engaging creative stories. Designers need to create stories that are both captivating and practically feasible when conducting case studies, site analysis, and scheme development. Therefore, the challenge lies in how to enable AI to narrate these stories from the designer's perspective and ensure they align with real-world needs. 2. **Reflecting Actual Conditions**: Design documents must consider the actual constraints and potentials of the site to ensure the innovation and feasibility of the design proposals. Thus, the second challenge for AI in architectural and environmental design is how to maintain innovation while ensuring the design proposals fit the actual conditions of the site. 3. **Cross-Modal Information Alignment**: All design strategies need to be accurately reflected in site images to ensure the practical executability of the design. This requires design proposals to include both engaging stories and visual descriptions that complement each other, enhancing the feasibility and implementability of the design. To address the above challenges, the researchers designed a comprehensive framework aimed at stimulating "innovative" thinking in large models while ensuring the "reliability" of these ideas. This framework achieves cross-modal information alignment through a text-based image generation model, ultimately generating a multimodal design proposal that users can further adjust.