PlantoGraphy: Incorporating Iterative Design Process into Generative Artificial Intelligence for Landscape Rendering

Rong Huang,Hai-Chuan Lin,Chuanzhang Chen,Kang Zhang,Wei Zeng

DOI: https://doi.org/10.1145/3613904.3642824

2024-01-30

Abstract:Landscape renderings are realistic images of landscape sites, allowing stakeholders to perceive better and evaluate design ideas. While recent advances in Generative Artificial Intelligence (GAI) enable automated generation of landscape renderings, the end-to-end methods are not compatible with common design processes, leading to insufficient alignment with design idealizations and limited cohesion of iterative landscape design. Informed by a formative study for comprehending design requirements, we present PlantoGraphy, an iterative design system that allows for interactive configuration of GAI models to accommodate human-centered design practice. A two-stage pipeline is incorporated: first, concretization module transforms conceptual ideas into concrete scene layouts with a domain-oriented large language model; and second, illustration module converts scene layouts into realistic landscape renderings using a fine-tuned low-rank adaptation diffusion model. PlantoGraphy has undergone a series of performance evaluations and user studies, demonstrating its effectiveness in landscape rendering generation and the high recognition of its interactive functionality.

Human-Computer Interaction

What problem does this paper attempt to address?

The paper attempts to address the issues primarily focused on the current inadequacies of generative artificial intelligence (GAI) tools in landscape design rendering. Although existing GAI tools can automatically generate landscape renderings, they typically operate in an end-to-end manner, lacking the flexibility compatible with designers' common design practices, especially in iterative design. Specifically, the paper identifies the following problems with existing GAI tools:

1. **Lack of interactive features to support iterative design**: Landscape design is an iterative process that requires continuous adjustment and refinement of design plans. However, existing GAI tools often fail to maintain consistency in results across multiple generations, making effective iterative design difficult.

2. **Insufficient sensitivity to the layout of design elements**: Landscape designers often use directional terms and common plant vocabulary to describe layouts when selecting plant species and determining optimal combinations. Existing text-to-image models struggle to accurately understand these descriptions. For example, descriptions like "daisies are below the dogwood, and white tulips are to the right of the daisies" may not be accurately parsed.

To address these issues, the paper proposes a system called PlantoGraphy, which has the following features:

- **Iterative design system**: PlantoGraphy allows users to configure GAI models interactively to fit human-centered design practices. The system includes two main modules: the Conceptualization Module and the Illustration Module.
- **Conceptualization Module**: Translates the designer's conceptual descriptions into specific scene layouts. This module introduces the concept of scene graphs, enhancing large language models' (LLM) understanding of landscape design descriptions through structured semantic descriptions.
- **Illustration Module**: Converts scene layouts into realistic landscape renderings. This module uses a diffusion model fine-tuned with Low-Rank Adaptation (LoRA) to improve the accuracy and precision of generating specific plants.

- **Interactive features**: The system provides a web-based user interface that allows users to easily generate landscape design renderings and iteratively optimize designs through interaction with the graph and layout components.

Through these innovations, PlantoGraphy aims to improve the graphical consistency of landscape design renderings, enhance the model's understanding of landscape design descriptions, and provide rich interactive editing features to better support designers' creative processes.