Abstract:Deep Reinforcement Learning (DRL) is a quickly evolving research field rooted in operations research and behavioural psychology, with potential applications extending across various domains, including robotics. This thesis delineates the background of modern Reinforcement Learning (RL), starting with the framework constituted by the Markov decision processes, Markov properties, goals and rewards, agent-environment interactions, and policies. We explain the main types of algorithms commonly used in RL, including value-based, policy gradient, and actor-critic methods, with a special emphasis on DQN, A2C and PPO. We then give a short literature review on some widely adopted frameworks for implementing RL algorithms and environments. Subsequently, we present Bidimensional Gripper Environment (BGE), a virtual simulator based on the Pymunk physics engine we developed to analyse top-down bidimensional object manipulation. The methodology section frames our agent-environment interaction as a Markov decision process, such that we can apply our RL algorithms. We list various goal formulation strategies, including reward shaping and curriculum learning. We also employ different steps of observation preprocessing to reduce the computational workload required. In the experimental phase, we run through a series of scenarios of increasing difficulty. We start with a simple static scenario and then gradually increase the amount of stochasticity. Whenever the agents show difficulty in learning, we counteract by increasing the degree of reward shaping and curriculum learning. These experiments demonstrate the substantial limitations and pitfalls of model-free algorithms under changing dynamics. In conclusion, we present a summary of our findings and remarks. We then outline potential future work to improve our methodology and possibly expand to real-world systems.

Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in First-person Simulated 3D Environments

Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in a First-person Simulated 3D Environment

Vision-Based Robotic Object Grasping—A Deep Reinforcement Learning Approach

Deep Reinforcement Learning for 2D Physics-Based Object Manipulation in Clutter

Learning Sparse Control Tasks from Pixels by Latent Nearest-Neighbor-Guided Explorations

Object-sensitive Deep Reinforcement Learning

VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learning

On the Efficacy of 3D Point Cloud Reinforcement Learning

The Ingredients of Real-World Robotic Reinforcement Learning

IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes

Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identifications

Active 6D Multi-Object Pose Estimation in Cluttered Scenarios with Deep Reinforcement Learning

VRKitchen: an Interactive 3D Environment for Learning Real Life Cooking Tasks

Visual Reinforcement Learning with Self-Supervised 3D Representations

Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation

Real-World Dexterous Object Manipulation based Deep Reinforcement Learning

Efficient Exploration and Discriminative World Model Learning with an Object-Centric Abstraction

Seeing by haptic glance: reinforcement learning-based 3D object Recognition

Deep Reinforcement Learning-Based DQN Agent Algorithm for Visual Object Tracking in a Virtual Environmental Simulation

Task-Induced Representation Learning

Dealing with Sparse Rewards in Reinforcement Learning