Abstract:The architectural environment is expanding into digital, virtual, and informational dimensions, introducing challenges in virtual-real space interaction. Traditional design methods struggle with real-time interaction, integration with existing workflows, and rapid space modification. To address these issues, we present a generative design method that enables symbiotic interaction between virtual and real spaces using Mixed Reality (MR) and Generative Artificial Intelligence (AI) technologies. We developed two approaches: one using the Rhino modeling platform and the other based on the Unity3D game engine, tailored to different application needs. User experience testing in exhibition, leisure, and residential spaces evaluated our method's effectiveness. Results showed significant improvements in design flexibility, interactive efficiency, and user satisfaction. In the exhibition scenario, the Unity3D-based method excelled in rapid design modifications and immersive experiences. Questionnaire data indicated that MR offers good visual comfort and higher immersion than VR, effectively supporting architects in interface and scale design. Clustering analysis of participants' position and gaze data revealed diverse behavioral patterns in the virtual-physical exhibition space, providing insights for optimizing spatial layouts and interaction methods. Our findings suggest that the generative AI-driven MR method simplifies traditional design processes by enabling real-time modification and interaction with spatial interfaces through simple verbal and motion interactions. This approach streamlines workflows by reducing steps like measuring, modeling, and rendering, while enhancing user engagement and creativity. Overall, this method offers new possibilities for experiential exhibition and architectural design, contributing to future environments where virtual and real spaces coexist seamlessly.

Intelligent Interaction in Mixed Reality

A Survey on 360° Images and Videos in Mixed Reality: Algorithms and Applications

Composition of virtual-real worlds and intelligence integration of human-computer in mixed reality

Survey on Light Field 3D Display Applied to Mixed Reality

Mixed Reality Entertainment and Art

Augmented Reality and AI: Enhancing Human-Computer Interaction in the Metaverse

Artificial Intelligence for Metaverse: A Framework

Interaction in Metaverse: A Survey

Multi-Player Immersive Communications and Interactions in Metaverse: Challenges, Architecture, and Future Directions

Human-Centered Interaction in Virtual Worlds: A New Era of Generative Artificial Intelligence and Metaverse

Intuitive Interactive Interface and Virtual Reality

Realizing the Metaverse with Edge Intelligence: A Match Made in Heaven

Enhancing Interaction in Virtual-Real Architectural Environments: A Comparative Analysis of Generative AI-driven Reality Approaches

Display,interactions and applications of immersive metaverse:Progress and outlooks

Research on the application of mixed reality technology in the field of art design

Mixed Reality: The Interface of the Future

Subtle Visual Cues in Mixed Reality: Influencing User Perception and Facilitating Interaction

Mixed Reality Interaction Techniques

Immersive virtual reality application for intelligent manufacturing: Applications and art design

Innovative Application and User Experience of Virtual Reality Technology in Human-Computer Interaction

Virtual-Reality Interpromotion Technology for Metaverse: A Survey