Abstract:Dynamic difficulty adjustment ($DDA$) is a process of automatically changing a game difficulty for the optimization of user experience. It is a vital part of almost any modern game. Most existing DDA approaches concentrate on the experience of a player without looking at the rest of the players. We propose a method that automatically optimizes user experience while taking into consideration other players and macro constraints imposed by the game. The method is based on deep neural network architecture that involves a count loss constraint that has zero gradients in most of its support. We suggest a method to optimize this loss function and provide theoretical analysis for its performance. Finally, we provide empirical results of an internal experiment that was done on $200,000$ players and was found to outperform the corresponding manual heuristics crafted by game design experts.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the dynamic adjustment of difficulty (Dynamic Difficulty Adjustment, DDA) in game design to optimize user experience (User Experience, UX). Specifically, most of the existing DDA methods mainly focus on the experience of a single player, without considering the influence of other players and the overall macro - constraints of the game. This paper proposes a new method, which can not only automatically optimize the user experience according to each player's ability, skills and behavior, but also consider the behavior of other players and the overall constraints of the game at the same time. ### Main contributions of the paper: 1. **UX loss function**: A new user experience (UX) loss function is proposed, which takes into account not only the experience of a single player, but also the experience of all other players. This method requires that the difficulty of a player conforms to both his or her personal style and the style of similar players. This loss function can be applied to various games and can learn how to define the optimal difficulty through neural networks. 2. **Completion rate constraint**: It shows how to use the completion rate constraint in neural networks. The completion rate refers to the proportion of players who complete a specific goal or a series of goals. It is a challenge to directly use the completion rate in the optimization process because the completion rate is a piecewise - constant function and its gradient is almost zero everywhere. For this reason, the author proposes a variant based on the projected gradient descent algorithm, which projects the parameters of the neural network onto the feasible set that satisfies the completion rate constraint through an iterative process. 3. **DDA system for practical applications**: A DDA system tested in an online game is introduced. This system is based on a deep neural network, optimizes the UX loss function, and operates under the completion rate constraint. The experimental results show that this system is superior to the method of manually setting difficulty in optimizing user experience. ### Technical details: - **UX loss function**: It is defined as: \[ \text{UX Loss}(\hat{D})=\text{var}(\hat{D})+\alpha\sum_{i = 0}^{M}(d_i-\hat{d}_i)^2 \] where $\text{var}(\hat{D})$ represents the variance of difficulty within the player group, $(d_i-\hat{d}_i)^2$ represents the difference between the actual difficulty and the predicted difficulty of a single player, and $\alpha$ is a parameter that controls the weights of the two. - **Completion rate constraint**: It is defined as: \[ \hat{P}\triangleq\frac{1}{M}\sum_{i = 1}^{M}1[d_i\geq\hat{d}_i]=P \] where $\hat{P}$ is the actual completion rate, $P$ is the desired completion rate, and $1[x]$ is an indicator function, which takes 1 when $x$ is true and 0 otherwise. - **Optimization method**: The neural network is trained by alternately optimizing the UX loss function and the completion rate constraint. The specific steps include: - Using the stochastic gradient descent method (SGD) to optimize the UX loss function. - Projecting the network weights onto the feasible set that satisfies the completion rate constraint. ### Experimental results: - **A/B testing**: An A/B test was carried out in an eight - day mini - game of a game. The test group used the machine - learning method to set the difficulty, and the control group used the rule - based method to set the difficulty. The results show that while maintaining the average completion rate, the machine - learning method significantly reduces the variance of the completion rate and improves the consistency of the user experience. ### Theoretical analysis: - **Convergence**: It is proved that in the process of alternately optimizing the UX loss function and the completion rate constraint, the algorithm can converge to a local optimal solution. Specifically, by alternately applying two projection operators (one for finding the nearest local minimum and the other for finding the point that satisfies the completion rate constraint), the convergence of the algorithm can be guaranteed. In conclusion, this paper proposes a new DDA method. By combining the UX loss function and the completion rate constraint, it can automatically optimize the user experience in large - scale online games while maintaining the fairness and stability of the game.

DL-DDA -- Deep Learning based Dynamic Difficulty Adjustment with UX and Gameplay constraints

General Dynamic Difficulty Adjustment System for Major Game Genres.

Novel Dynamic Difficulty Adjustment Methods for Niche Games

Exploring Dynamic Difficulty Adjustment in Videogames

Rethinking dynamic difficulty adjustment for video game design

Dynamic Difficulty Adjustment (DDA) in Computer Games: A Review

Dynamic difficulty adjustment approaches in video games: a systematic literature review

Dynamic difficulty adjustment using deep reinforcement learning: A review

Optimizing Player'S Satisfaction Through Dynamic Difficulty Adjustment Based On Amaf Heuristics For Game Dead End

Player Dominance Adjustment in Games

Colwell's Castle Defence: A Custom Game Using Dynamic Difficulty Adjustment to Increase Player Enjoyment

Continuous Reinforcement Learning-based Dynamic Difficulty Adjustment in a Visual Working Memory Game

Fast Game Content Adaptation Through Bayesian-based Player Modelling

A data-driven approach for online adaptation of game difficulty

AlphaDDA: strategies for adjusting the playing strength of a fully trained AlphaZero system to a suitable human training partner

Personalized Dynamic Difficulty Adjustment -- Imitation Learning Meets Reinforcement Learning

A Difficulty-Aware Framework for Churn Prediction and Intervention in Games

Dynamic Difficulty Adjustment on MOBA Games

Dynamic difficulty adjustment based on an improved algorithm of UCT for the Pac-Man Game

A Health Point-Based Dynamic Difficulty Adjustment Strategy for Video Games

Dynamic Difficulty Adjustment in Serious Games for Cerebral Visual Impairment