Abstract:In reinforcement learning, a reward function is a priori specified mapping that informs the learning agent how well its current actions and states are performing. From the viewpoint of training, reinforcement learning requires no labeled data and has none of the errors that are induced in supervised learning because responsibility is transferred from the loss function to the reward function. Methods that infer an approximated reward function using observations of demonstrations are termed inverse reinforcement learning or apprenticeship learning. A reward function is generated that reproduces observed behaviors. In previous studies, the reward function is implemented by estimating the maximum likelihood, Bayesian or information theoretic methods. This study proposes an inverse reinforcement learning method that has an approximated reward function as a linear combination of feature expectations, each of which plays a role in a base weak classifier. This approximated reward function is used by the agent to learn a policy, and the resultant behaviors are compared with an expert demonstration. The difference between the behaviors of the agent and those of the expert is measured using defined metrics, and the parameters for the approximated reward function are adjusted using an ensemble fuzzy method that has a boosting classification. After some interleaving iterations, the agent performs similarly to the expert demonstration. A fuzzy method is used to assign credits for the rewards in respect of the most recent decision to the neighboring states. Using the proposed method, the agent approximates the expert behaviors in fewer steps. The results of simulation demonstrate that the proposed method performs well in terms of sampling efficiency.

Online Sub-Sampling for Reinforcement Learning with General Function Approximation

Offline Reinforcement Learning with Differentiable Function Approximation is Provably Efficient

On the Role of General Function Approximation in Offline Reinforcement Learning.

Provably Efficient Reinforcement Learning with Linear Function Approximation

What can online reinforcement learning with function approximation benefit from general coverage conditions?

On Reward-Free Reinforcement Learning with Linear Function Approximation

Provably Efficient Reinforcement Learning with General Value Function Approximation.

A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximation

Toward General Function Approximation in Nonstationary Reinforcement Learning

What are the Statistical Limits of Offline RL with Linear Function Approximation?

Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learning

Understanding Deep Neural Function Approximation in Reinforcement Learning via $ε$-Greedy Exploration

An Ensemble Fuzzy Approach for Inverse Reinforcement Learning

Linear Bellman Completeness Suffices for Efficient Online Reinforcement Learning with Few Actions

Reinforcement Learning Algorithms with Function Approximation: Recent Advances and Applications

Non-stationary Reinforcement Learning under General Function Approximation

Uncertainty-Aware Reward-Free Exploration with General Function Approximation

On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyond

Posterior Sampling with Delayed Feedback for Reinforcement Learning with Linear Function Approximation

Efficient Online Reinforcement Learning with Offline Data