Abstract:The realization of an interoperable and scalable virtual platform, currently known as the “metaverse,” is inevitable, but many technological challenges need to be overcome first. With the metaverse still in a nascent phase, research currently indicates that building a new 3D social environment capable of interoperable avatars and digital transactions will represent most of the initial investment in time and capital. The return on investment, however, is worth the financial risk for firms like Meta, Google, and Apple. While the current virtual space of the metaverse is worth $6.30 billion, that is expected to grow to $84.09 billion by the end of 2028. But the creation of an entire alternate virtual universe of 3D avatars, objects, and otherworldly cityscapes calls for a new development pipeline and workflow. Existing 3D modeling and digital twin processes, already well-established in industry and gaming, will be ported to support the need to architect and furnish this new digital world. The current development pipeline, however, is cumbersome, expensive and limited in output capacity. This paper proposes a new and innovative immersive development pipeline leveraging the recent advances in artificial intelligence (AI) for 3D model creation and optimization. The previous reliance on 3D modeling software to create assets and then import into a game engine can be replaced with nearly instantaneous content creation with AI. While AI art generators like DALL-E 2 and DeepAI have been used for 2D asset creation, when combined with game engine technology, such as Unreal Engine 5 and virtualized geometry systems like Nanite, a new process for creating nearly unlimited content for immersive reality is possible. New processes and workflows, such as those proposed here, will revolutionize content creation and pave the way for Web 3.0, the metaverse and a truly 3D social environment.

Towards a Holodeck-style Simulation Game

Unbounded: A Generative Infinite Game of Character Life Simulation

Holodeck: Language Guided Generation of 3D Embodied AI Environments

Language as Reality: A Co-Creative Storytelling Game Experience in 1001 Nights using Generative AI

Humanoid Agents: Platform for Simulating Human-like Generative Agents

Using Unity to Help Solve Intelligence

Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generation

Scaling Instructable Agents Across Many Simulated Worlds

InfiniteWorld: A Unified Scalable Simulation Framework for General Visual-Language Robot Interaction

Project Thyia: A Forever Gameplayer

Generating Interactive Worlds with Text

Open-source 3D discrete event simulator based on the game engine unity

Heter-Sim: Heterogeneous Multi-Agent Systems Simulation by Interactive Data-Driven Optimization

A Proposed Meta-Reality Immersive Development Pipeline: Generative AI Models and Extended Reality (XR) Content for the Metaverse

Unity: A General Platform for Intelligent Agents

Imagined Potential Games: A Framework for Simulating, Learning and Evaluating Interactive Behaviors

High-Fidelity Simulated Players for Interactive Narrative Planning

Generative Agents: Interactive Simulacra of Human Behavior

JaxLife: An Open-Ended Agentic Simulator

InfiniCity: Infinite-Scale City Synthesis

Learning Interactive Real-World Simulators