Playable Game Generation

Mingyu Yang,Junyou Li,Zhongbin Fang,Sheng Chen,Yangbin Yu,Qiang Fu,Wei Yang,Deheng Ye

2024-12-02

Abstract:In recent years, Artificial Intelligence Generated Content (AIGC) has advanced from text-to-image generation to text-to-video and multimodal video synthesis. However, generating playable games presents significant challenges due to the stringent requirements for real-time interaction, high visual quality, and accurate simulation of game mechanics. Existing approaches often fall short, either lacking real-time capabilities or failing to accurately simulate interactive mechanics. To tackle the playability issue, we propose a novel method called \emph{PlayGen}, which encompasses game data generation, an autoregressive DiT-based diffusion model, and a comprehensive playability-based evaluation framework. Validated on well-known 2D and 3D games, PlayGen achieves real-time interaction, ensures sufficient visual quality, and provides accurate interactive mechanics simulation. Notably, these results are sustained even after over 1000 frames of gameplay on an NVIDIA RTX 2060 GPU. Our code is publicly available: <a class="link-external link-https" href="https://github.com/GreatX3/Playable-Game-Generation" rel="external noopener nofollow">this https URL</a>. Our playable demo generated by AI is: <a class="link-external link-http" href="http://124.156.151.207" rel="external noopener nofollow">this http URL</a>.

Artificial Intelligence

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is the challenges in Playable Game Generation, especially how to ensure that the generated games have real - time interaction capabilities, sufficient visual quality, and accurate simulation of interaction mechanisms. Specifically:

1. **Real - time Interaction**: The generated games need to be able to respond to players' input commands, and these interactions must be real - time to provide a smooth gaming experience.
2. **Sufficient Visual Quality**: The generated game frames should maintain the same level of detail, color accuracy, and overall visual fidelity as the original game.
3. **Accurate Simulation of Interaction Mechanisms**: The behaviors of characters in the game should accurately reflect the physical rules and interaction mechanisms of the game.

Existing methods have deficiencies in these aspects:
- Lack of real - time interaction capabilities (for example, only being able to handle a limited number of frames or lacking meaningful action representations).
- Unable to accurately simulate interaction mechanisms, resulting in significant hallucination phenomena during long - term gaming.

To solve these problems, the author proposes a new method called PlayGen, and its main contributions include:

- **Playability Workflow Development**: PlayGen can generate playable games with real - time interaction capabilities, sufficient visual quality, and accurate simulation of interaction mechanisms.
- **Automated Evaluation Method**: Developed a set of automated evaluation systems that can evaluate the generation efficiency, visual quality, and accuracy of interaction mechanism simulation of game generation models.
- **Benchmark Testing**: Conducted extensive tests on well - known 2D and 3D game benchmarks, proving the effectiveness and reliability of this method.

Through these improvements, PlayGen can achieve 20 FPS real - time interaction on consumer - level graphics cards (such as NVIDIA RTX 2060), and maintain accurate simulation of interaction mechanisms during the game process of more than 1,000 frames. In addition, this method can also balance visual quality and frame rate when adjusting parameters without sacrificing the accuracy of interaction mechanisms.

### Method Overview

The method of PlayGen mainly consists of three parts:

1. **Data Generation**: Collect diverse game data to ensure wide coverage of game scenarios, and make the data distribution more balanced through the clustering sampling method.
2. **Model Training and Inference**: Use Variational Auto - Encoder (VAE) and Latent Diffusion Model (LDM) to learn the game state transition model, and improve the simulation accuracy of rare interactions through the self - supervised long - tail transition learning method.
3. **Model Evaluation**: Introduce playability - based evaluation methods to quantitatively evaluate the interaction efficiency, visual quality, and accuracy of interaction mechanism simulation.

Through these strategies, PlayGen successfully solves the main challenges encountered by existing methods in generating playable games.