Abstract:One of the challenges in self-adaptive systems concerns how to make adaptation to themselves at runtime in response to possible and even unexpected changes from the environment and/or user goals. A feasible solution to this challenge is rule-based adaptation, in which, adaptation decisions are made according to predefined rules that specify what particular actions should be performed to react to different changing events from the environment. Although it has the characteristic of highly- efficient decision making for adaptation, rule-based adaptation has two limitations: 1. no guarantee that those predefined rules will lead to optimal or nearly-optimal adaptation results; 2. weak support to evolve these rules to cope with non-stationary environment and changeable user goals at runtime. In this paper, we propose a reinforcement learning-based framework to the generation and evolution of software adaptation rules. This framework manifests two key capabilities for self-adaptation: 1. the capability of automatically learning adaptation rules from different goal settings at the offline phase; 2. the capability of automatically evolving adaptation rules from real-time information about the environment and user goals at the online phase. The two capabilities are built on the combination of reinforcement learning and case-based reasoning techniques. This framework improves the existing rule-based adaptation from two points: the flexibility of adaptation logic, and the quality of adaptation rules. We evaluate this framework through a case study of an E-commerce web application, which shows that this framework improves both the efficiency and effectiveness of self-adaptation.

Cross-Platform Software Framework for Self-Adaptive Mobile Robot Systems

Mobile Robot Middleware Supporting Self-adaptive Programming

Constructing Self-Adaptive Software Systems: a Case Study with an Obstacle-Avoiding Mini-Car

A Self-Adaptation Framework for Dealing with the Complexities of Software Changes

Self-adaptive Software: Cybernetic Perspective and an Application Server Supported Framework

A General-Purpose Application Platform For Multiple Heterogeneous Mobile Robots

An Adaptive Component Model for Autonomous Mobile Robots

A Middleware-Based Simulation and Control Framework for Mobile Service Robots

Web-based Robotic Control System with Flexible Framework

Towards a General Supporting Framework for Self-Adaptive Software Systems.

Framework Of Industrial Robot System Programming And Management Software

ALLIANCE-ROS: A Software Framework on ROS for Fault-Tolerant and Cooperative Mobile Robots

A Reinforcement Learning-Based Framework for the Generation and Evolution of Adaptation Rules

User Preference Based Autonomic Generation of Self-Adaptive Rules

Component-based Software Framework for Network Mobile Robot

Self-adaptive Software Model and Platform Implementation for Ubiquitous Computing

The Design of One Cross-platform Multimodal Interactive Control System

A Component Based Hybrid Embedded Framework for Mobile Robot

An Embedded Platform for Intelligent Mobile Robot

Hierarchical Needs Based Self-Adaptive Framework For Cooperative Multi-Robot System

Development of a Component-based Mobile Robot Software System