Abstract:Computational agents support humans in many areas of life and are therefore found in heterogeneous contexts. This means that agents operate in rapidly changing environments and can be confronted with huge state and action spaces. In order to perform services and carry out activities satisfactorily, i.e. in a goal-oriented manner, agents require prior knowledge and therefore have to develop and pursue context-dependent policies. The problem here is that prescribing policies in advance is limited and inflexible, especially in dynamically changing environments. Moreover, the context (i.e. the external and internal state) of an agent determines its choice of actions. Since the environments in which agents operate can be stochastic and complex in terms of the number of states and feasible actions, activities are usually modelled in a simplified way by Markov decision processes so that, for example, agents with reinforcement learning are able to learn policies, i.e. state-action pairs, that help to capture the context and act accordingly to optimally perform activities. However, training policies for all possible contexts using reinforcement learning is time-consuming. A requirement and challenge for agents is to learn strategies quickly and respond immediately in cross-context environments and applications, e.g., the Internet, service robotics, cyber-physical systems. In this work, we propose a novel simulation-based approach that enables a) the representation of heterogeneous contexts through knowledge graphs and entity embeddings and b) the context-aware composition of policies on demand by ensembles of agents running in parallel. The evaluation we conducted with the "Virtual Home" dataset indicates that agents with a need to switch seamlessly between different contexts, e.g. in a home environment, can request on-demand composed policies that lead to the successful completion of context-appropriate activities without having to learn these policies in lengthy training steps and episodes, in contrast to agents that use reinforcement learning. The presented approach enables both context-aware and cross-context applicability of untrained computational agents. Furthermore, the source code of the approach as well as the generated data, i.e. the trained embeddings and the semantic representation of domestic activities, is open source and openly accessible on Github and Figshare.

Context Discovery for Model Learning in Partially Observable Environments

Discovering Latent States for Model Learning: Applying Sensorimotor Contingencies Theory and Predictive Processing to Model Context

Context Exploitation using Hierarchical Bayesian Models

Learning to Make Predictions In Partially Observable Environments Without a Generative Model

Safe Reinforcement Learning in Uncertain Contexts

Adapting to Unseen Environments through Explicit Representation of Context

Explore the Context: Optimal Data Collection for Context-Conditional Dynamics Models

Online Context Learning for Socially-compliant Navigation

An IoT Analytics Embodied Agent Model based on Context-Aware Machine Learning

Cross-context Visual Imitation Learning from Demonstrations.

Learning to Look Around: Intelligently Exploring Unseen Environments for Unknown Tasks

Multi-Modal Imitation Learning in Partially Observable Environments

CogExplore: Contextual Exploration with Language-Encoded Environment Representations

Inferring Belief States in Partially-Observable Human-Robot Teams

A Contextual Bandit Approach for Learning to Plan in Environments with Probabilistic Goal Configurations

Context-aware composition of agent policies by Markov decision process entity embeddings and agent ensembles

Human Activity Learning and Segmentation using Partially Hidden Discriminative Models

Generalization of Agent Behavior through Explicit Representation of Context