Evolving Multimodal Behavior

Jacob Schrum
Abstract:Multimodal behavior occurs when an agent exhibits distinctly different kinds of actions under different circumstances. Many interesting problems in real and simulated environments require agents that exhibit such behavior. The ability to automatically discover multimodal behavior would be useful in robotics, video games and other high-level control problems. Multimodal behavior is also especially important for teams of agents, taking the form of division of labor between team members. This proposed dissertation develops a method for discovering such behavior via neuroevolution. Work completed so far demonstrates how three modifications to typical neuroevolutionary methods make multimodal behavior easier to evolve: (1) multiobjective evolution (via e.g. the multiobjective evolutionary algorithm NSGA-II) encourages multimodal behavior because distinct behaviors tend to be associated with sets of contradictory objectives, (2) whenever the population collectively surpasses preset objective goals, the corresponding objectives can be dropped, speeding up evolution, and (3) a special mutation operator that creates a new set of output neurons for a neural network encourages the development of multiple distinct behavioral modes. The proposed work will build upon these findings in order to improve the evolution of multimodal behavior further in the following ways: (1) methods of overcoming stagnation via behavioral diversity enhancement will be developed, (2) the new-mode mutation will be improved, and different methods of arbitrating between the multiple modes evaluated, (3) in order to evolve teams more effectively, populations will be divided into subpopulations for each role in the team, and (4) the dynamic objective dropping mechanism will be modified to be an open-ended learning process. The resulting algorithm will be evaluated in a set of increasingly challenging multimodal domains, including Unreal Tournament 2004, a complex commercial first-person shooter video game. Success in these domains will demonstrate the algorithm’s ability to evolve interesting behavior for challenging domains.
What problem does this paper attempt to address?