Abstract:The aim of Reinforcement Learning (RL) in real-world applications is to create systems capable of making autonomous decisions by learning from their environment through trial and error. This paper emphasizes the importance of reward engineering and reward shaping in enhancing the efficiency and effectiveness of reinforcement learning algorithms. Reward engineering involves designing reward functions that accurately reflect the desired outcomes, while reward shaping provides additional feedback to guide the learning process, accelerating convergence to optimal policies. Despite significant advancements in reinforcement learning, several limitations persist. One key challenge is the sparse and delayed nature of rewards in many real-world scenarios, which can hinder learning progress. Additionally, the complexity of accurately modeling real-world environments and the computational demands of reinforcement learning algorithms remain substantial obstacles. On the other hand, recent advancements in deep learning and neural networks have significantly improved the capability of reinforcement learning systems to handle high-dimensional state and action spaces, enabling their application to complex tasks such as robotics, autonomous driving, and game playing. This paper provides a comprehensive review of the current state of reinforcement learning, focusing on the methodologies and techniques used in reward engineering and reward shaping. It critically analyzes the limitations and recent advancements in the field, offering insights into future research directions and potential applications in various domains.

Toward Computationally Efficient Inverse Reinforcement Learning via Reward Shaping

Inverse Reinforcement Learning with Unknown Reward Model based on Structural Risk Minimization

Convergence Analysis of an Incremental Approach to Online Inverse Reinforcement Learning

Learning to Utilize Shaping Rewards: A New Approach of Reward Shaping

Learning to Shape Rewards Using a Game of Two Partners

Basis for Intentions: Efficient Inverse Reinforcement Learning using Past Experience

ORSO: Accelerating Reward Design via Online Reward Selection and Policy Optimization

Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applications

Reward Shaping via Meta-Learning

Barrier Functions Inspired Reward Shaping for Reinforcement Learning

Learning to Shape Rewards using a Game of Switching Controls

Towards Theoretical Understanding of Inverse Reinforcement Learning

Shaping Reward Learning Approach from Passive Samples

Benchmarking Potential Based Rewards for Learning Humanoid Locomotion

Curricular Subgoals for Inverse Reinforcement Learning

Potential-Based Reward Shaping For Intrinsic Motivation

A Framework and Method for Online Inverse Reinforcement Learning

Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignment

Understanding Reward Ambiguity Through Optimal Transport Theory in Inverse Reinforcement Learning

On the Effective Horizon of Inverse Reinforcement Learning