Abstract:One of the challenges in self-adaptive systems concerns how to make adaptation to themselves at runtime in response to possible and even unexpected changes from the environment and/or user goals. A feasible solution to this challenge is rule-based adaptation, in which, adaptation decisions are made according to predefined rules that specify what particular actions should be performed to react to different changing events from the environment. Although it has the characteristic of highly- efficient decision making for adaptation, rule-based adaptation has two limitations: 1. no guarantee that those predefined rules will lead to optimal or nearly-optimal adaptation results; 2. weak support to evolve these rules to cope with non-stationary environment and changeable user goals at runtime. In this paper, we propose a reinforcement learning-based framework to the generation and evolution of software adaptation rules. This framework manifests two key capabilities for self-adaptation: 1. the capability of automatically learning adaptation rules from different goal settings at the offline phase; 2. the capability of automatically evolving adaptation rules from real-time information about the environment and user goals at the online phase. The two capabilities are built on the combination of reinforcement learning and case-based reasoning techniques. This framework improves the existing rule-based adaptation from two points: the flexibility of adaptation logic, and the quality of adaptation rules. We evaluate this framework through a case study of an E-commerce web application, which shows that this framework improves both the efficiency and effectiveness of self-adaptation.

Rulebook : An Architectural Pattern for Self-Amending Mechanics in Digital Games

Process Rule Abstraction and Storage of Collaborative Pattern Design

Automatic Game Design via Mechanic Generation

Self-organized evolution of mechanism kinetic scheme based on axiomatic design

Towards Self-Adaptive Game Logic

Mechanic Maker 2.0: Reinforcement Learning for Evaluating Generated Rules

Rule-based Method for Generating Mechanical Kinematic Function Scheme

Design and Evaluation of Parametrizable Multi-genre Game Mechanics

A Data Driven Review of Board Game Design and Interactions of Their Mechanics

A Controllable Co-Creative Agent for Game System Design

Don't Flip the Table Yet: A Framework for the Analysis of Visual and Cognitive Ergonomics in Board Games

Game Play Schemas: From Player Analysis to Adaptive Game Mechanics

Design of Configurable Rule Engine for Information Filtering

Liability, Ethics, and Culture-Aware Behavior Specification using Rulebooks

Rule switching mechanisms in the Game of Life with synchronous and asynchronous updating policy

RulePad: Interactive Authoring of Checkable Design Rules

A Reinforcement Learning-Based Framework for the Generation and Evolution of Adaptation Rules

Game Mechanics in the Design of a Collaborative 3D Serious Game

Identifying Game Mechanics for Integrating Fabrication Activities within Existing Digital Games

Dynamic Social Interaction Mechanics CrossAnt

From Rules to Runs:A Dynamic Epistemic Take on Imperfect Information Games