Abstract:Recently, machine learning (ML) has become a popular approach to support self-adaptation. ML has been used to deal with several problems in self-adaptation, such as maintaining an up-to-date runtime model under uncertainty and scalable decision-making. Yet, exploiting ML comes with inherent challenges. In this paper, we focus on a particularly important challenge for learning-based self-adaptive systems: drift in adaptation spaces. With adaptation space we refer to the set of adaptation options a self-adaptive system can select from at a given time to adapt based on the estimated quality properties of the adaptation options. Drift of adaptation spaces originates from uncertainties, affecting the quality properties of the adaptation options. Such drift may imply that eventually no adaptation option can satisfy the initial set of the adaptation goals, deteriorating the quality of the system, or adaptation options may emerge that allow enhancing the adaptation goals. In ML, such shift corresponds to novel class appearance, a type of concept drift in target data that common ML techniques have problems dealing with. To tackle this problem, we present a novel approach to self-adaptation that enhances learning-based self-adaptive systems with a lifelong ML layer. We refer to this approach as lifelong self-adaptation. The lifelong ML layer tracks the system and its environment, associates this knowledge with the current tasks, identifies new tasks based on differences, and updates the learning models of the self-adaptive system accordingly. A human stakeholder may be involved to support the learning process and adjust the learning and goal models. We present a general architecture for lifelong self-adaptation and apply it to the case of drift of adaptation spaces that affects the decision-making in self-adaptation. We validate the approach for a series of scenarios using the DeltaIoT exemplar.

Analysis of hidden feedback loops in continuous machine learning systems

A Mathematical Model of the Hidden Feedback Loop Effect in Machine Learning Systems

A Classification of Feedback Loops and Their Relation to Biases in Automated Decision-Making Systems

Feedback Loops With Language Models Drive In-Context Reward Hacking

Data Feedback Loops: Model-driven Amplification of Dataset Biases

Does Machine Learning Amplify Pricing Errors in the Housing Market? -- The Economics of Machine Learning Feedback Loops

Continual Learning in Human Activity Recognition: an Empirical Analysis of Regularization

Human-Algorithm Interaction Biases in the Big Data Cycle: A Markov Chain Iterated Learning Framework

Competing AI: How does competition feedback affect machine learning?

N A ] 2 6 Se p 20 20 Machine Learning from a Continuous Viewpoint

Learning Interpretable Concept-Based Models with Human Feedback

Looping In: Exploring Feedback Strategies to Motivate Human Engagement in Interactive Machine Learning

Characterizing Concept Drift

Emergent Instabilities in Algorithmic Feedback Loops

Dealing with Drift of Adaptation Spaces in Learning-based Self-Adaptive Systems using Lifelong Self-Adaptation

Handling Concept Drifts in Regression Problems -- the Error Intersection Approach

Automating concept-drift detection by self-evaluating predictive model degradation

Understanding Feedback Mechanisms in Machine Learning Jupyter Notebooks

Concept Drift Adaptation with Continuous Kernel Learning

Machine Learning from a Continuous Viewpoint,i

Identifying Predictions That Influence the Future: Detecting Performative Concept Drift in Data Streams