Abstract:The architectural environment is expanding into digital, virtual, and informational dimensions, introducing challenges in virtual-real space interaction. Traditional design methods struggle with real-time interaction, integration with existing workflows, and rapid space modification. To address these issues, we present a generative design method that enables symbiotic interaction between virtual and real spaces using Mixed Reality (MR) and Generative Artificial Intelligence (AI) technologies. We developed two approaches: one using the Rhino modeling platform and the other based on the Unity3D game engine, tailored to different application needs. User experience testing in exhibition, leisure, and residential spaces evaluated our method's effectiveness. Results showed significant improvements in design flexibility, interactive efficiency, and user satisfaction. In the exhibition scenario, the Unity3D-based method excelled in rapid design modifications and immersive experiences. Questionnaire data indicated that MR offers good visual comfort and higher immersion than VR, effectively supporting architects in interface and scale design. Clustering analysis of participants' position and gaze data revealed diverse behavioral patterns in the virtual-physical exhibition space, providing insights for optimizing spatial layouts and interaction methods. Our findings suggest that the generative AI-driven MR method simplifies traditional design processes by enabling real-time modification and interaction with spatial interfaces through simple verbal and motion interactions. This approach streamlines workflows by reducing steps like measuring, modeling, and rendering, while enhancing user engagement and creativity. Overall, this method offers new possibilities for experiential exhibition and architectural design, contributing to future environments where virtual and real spaces coexist seamlessly.

Poster: A hybrid direct visual editing method for architectural massing study in virtual environments

Using the CAT for 3D Sketching in Front of Large Displays

Immersive Display and Interactive Techniques for the Modeling and Rendering of Virtual Heritage Environments

Designing a 3D modelling tool for novice users.

Exploitation of Multiplayer Interaction and Development of Virtual Puppetry Storytelling Using Gesture Control and Stereoscopic Devices.

A touchable virtual screen interaction system with handheld Kinect camera

4Doodle: Two-handed Gestures for Immersive Sketching of Architectural Models

An Approach to Solid Modeling in a Semi-Immersive Virtual Environment.

An Integrated Virtual Design And Virtual Assembly Environment

Virtual-Physical Interaction System

Combining Gesture and Voice Control for Mid-Air Manipulation of CAD Models in VR Environments

Interactive Sculpting of Product Shape Based on Constraint Manipulation in Virtual Environment

Elicitation and Evaluation of Hand-based Interaction Language for 3D Conceptual Design in Mixed Reality

Tangible Mixed Reality Desktop for Digital Media Management

Dual Quaternion Based Virtual Hand Interaction Modeling

Behavior Mapping Of Sketching In Vr Space With Physical Tablet Interface

VirtualDesk: A Comfortable and Efficient Immersive Information Visualization Approach

VR-CAD Framework for Parametric Data Modification with a 3D Shape-based Interaction

An evaluation of multimodal interaction techniques for 3D layout constraint solver in a desktop-based virtual environment

Vision‐tangible interactive display method for mixed and virtual reality: Toward the human‐centered editable reality

Enhancing Interaction in Virtual-Real Architectural Environments: A Comparative Analysis of Generative AI-driven Reality Approaches