Abstract:Multi-objective Markov decision processes are sequential decision-making problems that involve multiple conflicting reward functions that cannot be optimized simultaneously without a compromise. This type of problems cannot be solved by a single optimal policy as in the conventional case. Alternatively, multi-objective reinforcement learning methods evolve a coverage set of optimal policies that can satisfy all possible preferences in solving the problem. However, many of these methods cannot generalize their coverage sets to work in non-stationary environments. In these environments, the parameters of the state transition and reward distribution vary over time. This limitation results in significant performance degradation for the evolved policy sets. In order to overcome this limitation, there is a need to learn a generic skill set that can bootstrap the evolution of the policy coverage set for each shift in the environment dynamics therefore, it can facilitate a continuous learning process. In this work, intrinsically motivated reinforcement learning has been successfully deployed to evolve generic skill sets for learning hierarchical policies to solve multi-objective Markov decision processes. We propose a novel dual-phase intrinsically motivated reinforcement learning method to address this limitation. In the first phase, a generic set of skills is learned. While in the second phase, this set is used to bootstrap policy coverage sets for each shift in the environment dynamics. We show experimentally that the proposed method significantly outperforms state-of-the-art multi-objective reinforcement methods in a dynamic robotics environment.

Sub-policy Adaptation for Hierarchical Reinforcement Learning

Hierarchical Skills for Efficient Exploration

Hierarchical Reinforcement Learning with Advantage-Based Auxiliary Rewards

Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learning

Bidirectional-Reachable Hierarchical Reinforcement Learning with Mutually Responsive Policies

Temporal-adaptive Hierarchical Reinforcement Learning

Algorithms for Batch Hierarchical Reinforcement Learning

Past Data-Driven Adaptation in Hierarchical Reinforcement Learning

$\epsilon$-Invariant Hierarchical Reinforcement Learning for Building Generalizable Policy

Hierarchical Reinforcement Learning Based on Continuous Subgoal Space

Hierarchical Policy Learning is Sensitive to Goal Space Design

Relabeling and policy distillation of hierarchical reinforcement learning

Probabilistic Subgoal Representations for Hierarchical Reinforcement learning

Offline Hierarchical Reinforcement Learning via Inverse Optimization

Diversity-Driven Extensible Hierarchical Reinforcement Learning.

Efficient Hierarchical Policy Network with Fuzzy Rules

Policy Transfer Via Skill Adaptation and Composition

Abstract Value Iteration for Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning with Timed Subgoals

Intrinsically Motivated Hierarchical Policy Learning in Multi-objective Markov Decision Processes

Optimal Hierarchical Learning Path Design with Reinforcement Learning