Abstract:Artificial intelligence (AI) has achieved superhuman performance in board games such as Go, chess, and Othello (Reversi). In other words, the AI system surpasses the level of a strong human expert player in such games. In this context, it is difficult for a human player to enjoy playing the games with the AI. To keep human players entertained and immersed in a game, the AI is required to dynamically balance its skill with that of the human player. To address this issue, we propose AlphaDDA, an AlphaZero-based AI with dynamic difficulty adjustment (DDA). AlphaDDA consists of a deep neural network (DNN) and a Monte Carlo tree search, as in AlphaZero. AlphaDDA learns and plays a game the same way as AlphaZero, but can change its skills. AlphaDDA estimates the value of the game state from only the board state using the DNN. AlphaDDA changes a parameter dominantly controlling its skills according to the estimated value. Consequently, AlphaDDA adjusts its skills according to a game state. AlphaDDA can adjust its skill using only the state of a game without any prior knowledge regarding an opponent. In this study, AlphaDDA plays Connect4, Othello, and 6x6 Othello with other AI agents. Other AI agents are AlphaZero, Monte Carlo tree search, the minimax algorithm, and a random player. This study shows that AlphaDDA can balance its skill with that of the other AI agents, except for a random player. AlphaDDA can weaken itself according to the estimated value. However, AlphaDDA beats the random player because AlphaDDA is stronger than a random player even if AlphaDDA weakens itself to the limit. The DDA ability of AlphaDDA is based on an accurate estimation of the value from the state of a game. We believe that the AlphaDDA approach for DDA can be used for any game AI system if the DNN can accurately estimate the value of the game state and we know a parameter controlling the skills of the AI system.

Dynamic difficulty for checkers and Chinese chess

Hybrid Minimax-MCTS and Difficulty Adjustment for General Game Playing

Finding Game Levels with the Right Difficulty in a Few Trials through Intelligent Trial-and-Error

Dynamic Difficulty Adjustment on MOBA Games

Player Dominance Adjustment in Games

Advances in finding ideal play on poset games

Towards Understanding Chinese Checkers with Heuristics, Monte Carlo Tree Search, and Deep Reinforcement Learning

On the complexity of Dark Chinese Chess

Difficulty-skill balance does not affect engagement and enjoyment: a pre-registered study using artificial intelligence-controlled difficulty

Colwell's Castle Defence: A Custom Game Using Dynamic Difficulty Adjustment to Increase Player Enjoyment

A Health Point-Based Dynamic Difficulty Adjustment Strategy for Video Games

DL-DDA -- Deep Learning based Dynamic Difficulty Adjustment with UX and Gameplay constraints

Personalized Dynamic Difficulty Adjustment -- Imitation Learning Meets Reinforcement Learning

Implementation of Alpha-Beta Pruning and Transposition Tables on Checkers Game

AlphaZero Gomoku

On game psychology: an experiment on the chess board/screen, should you always "do your best", and why the programs with prescribed weaknesses cannot be our good friends?

A data-driven approach for online adaptation of game difficulty

AlphaDDA: strategies for adjusting the playing strength of a fully trained AlphaZero system to a suitable human training partner

Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess

Design and improvement of the pruning algorithm of the Chinese chess in the computer games

Exploring Dynamic Difficulty Adjustment in Videogames