Abstract:Multi-objective Markov decision processes are sequential decision-making problems that involve multiple conflicting reward functions that cannot be optimized simultaneously without a compromise. This type of problems cannot be solved by a single optimal policy as in the conventional case. Alternatively, multi-objective reinforcement learning methods evolve a coverage set of optimal policies that can satisfy all possible preferences in solving the problem. However, many of these methods cannot generalize their coverage sets to work in non-stationary environments. In these environments, the parameters of the state transition and reward distribution vary over time. This limitation results in significant performance degradation for the evolved policy sets. In order to overcome this limitation, there is a need to learn a generic skill set that can bootstrap the evolution of the policy coverage set for each shift in the environment dynamics therefore, it can facilitate a continuous learning process. In this work, intrinsically motivated reinforcement learning has been successfully deployed to evolve generic skill sets for learning hierarchical policies to solve multi-objective Markov decision processes. We propose a novel dual-phase intrinsically motivated reinforcement learning method to address this limitation. In the first phase, a generic set of skills is learned. While in the second phase, this set is used to bootstrap policy coverage sets for each shift in the environment dynamics. We show experimentally that the proposed method significantly outperforms state-of-the-art multi-objective reinforcement methods in a dynamic robotics environment.

Robust and Efficient Transfer Learning with Hidden-Parameter Markov Decision Processes

HiT-MDP: Learning the SMDP option framework on MDPs with Hidden Temporal Variables

HiT-MDP: Learning the SMDP option framework on MDPs with Hidden Temporal Embeddings

Sufficient Markov Decision Processes with Alternating Deep Neural Networks

Robust Anytime Learning of Markov Decision Processes

Robust Markov Decision Processes without Model Estimation

Hidden Parameter Recurrent State Space Models For Changing Dynamics Scenarios

Robust Markov Decision Processes: A Place Where AI and Formal Methods Meet

Bayes-adaptive hierarchical MDPs

Higher-Order Generalization Bounds: Learning Deep Probabilistic Programs via PAC-Bayes Objectives

Bayesian Nonparametric Hidden Semi-Markov Models

HiPPO: Recurrent Memory with Optimal Polynomial Projections

Transition Constrained Bayesian Optimization via Markov Decision Processes

Variational Hidden Conditional Random Fields with Beta Processes.

Multi-fidelity Hierarchical Neural Processes

Intrinsically Motivated Hierarchical Policy Learning in Multi-objective Markov Decision Processes

Hierarchical Average-Reward Linearly-solvable Markov Decision Processes

Lipschitz Lifelong Reinforcement Learning

Hidden Markov model using Dirichlet process for de-identification

Hindsight Learning for MDPs with Exogenous Inputs

Physics-guided Deep Markov Models for Learning Nonlinear Dynamical Systems with Uncertainty