Abstract:Due to the empirical success of reinforcement learning, an increasing number of students study the subject. However, from our practical teaching experience, we see students entering the field (bachelor, master and early PhD) often struggle. On the one hand, textbooks and (online) lectures provide the fundamentals, but students find it hard to translate between equations and code. On the other hand, public codebases do provide practical examples, but the implemented algorithms tend to be complex, and the underlying test environments contain multiple reinforcement learning challenges at once. Although this is realistic from a research perspective, it often hinders educational conceptual understanding. To solve this issue we introduce EduGym, a set of educational reinforcement learning environments and associated interactive notebooks tailored for education. Each EduGym environment is specifically designed to illustrate a certain aspect/challenge of reinforcement learning (e.g., exploration, partial observability, stochasticity, etc.), while the associated interactive notebook explains the challenge and its possible solution approaches, connecting equations and code in a single document. An evaluation among RL students and researchers shows 86% of them think EduGym is a useful tool for reinforcement learning education. All notebooks are available from <a class="link-external link-https" href="https://www.edugym.org/" rel="external noopener nofollow">this https URL</a>, while the full software package can be installed from <a class="link-external link-https" href="https://github.com/RLG-Leiden/edugym" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is a key gap in reinforcement learning education. Although there are many teaching materials for reinforcement learning, such as textbooks, online courses, and public code repositories, students still face challenges when entering this field. Specifically: 1. **Disconnection between theory and practice**: Textbooks and online lectures provide the basic knowledge of reinforcement learning, but it is difficult for students to transform these theoretical knowledge into actual code implementation. The lack of interactive practical examples makes many students feel confused or lengthens the learning cycle. 2. **High complexity**: Public code repositories provide practical examples, but the implemented algorithms are usually very complex, and the test environment also contains multiple reinforcement learning challenges. Although this complexity is realistic in research, it hinders the understanding of concepts in education. To solve these problems, the author introduced EduGym, which is a reinforcement learning environment and an interactive notebook suite specially designed for education. The main features of EduGym include: - **Specific educational environment**: Each EduGym environment is specially designed to illustrate a specific aspect or challenge of reinforcement learning (such as exploration, partial observability, randomness, etc.), and has adjustable parameters to adjust the intensity of a specific challenge. - **Interactive notebook**: Each environment is equipped with an interactive notebook, which explains the challenges and their possible solutions, and connects formulas and codes in one document, enabling students to conduct interactive experiments. In this way, EduGym aims to be a supplement to other reinforcement learning teaching materials, helping students better understand and master the concepts and practices of reinforcement learning.

EduGym: An Environment and Notebook Suite for Reinforcement Learning Education

Gymnasium: A Standard Interface for Reinforcement Learning Environments

LapGym -- An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery

Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms

Gym-ANM: Open-source software to leverage reinforcement learning for power system management in research and education

SDGym: Low-Code Reinforcement Learning Environments using System Dynamics Models

OR-Gym: A Reinforcement Learning Library for Operations Research Problems

ChemGymRL: An Interactive Framework for Reinforcement Learning for Digital Chemistry

Hearts Gym: Learning Reinforcement Learning as a Team Event

DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback

OffWorld Gym: open-access physical robotics environment for real-world reinforcement learning benchmark and research

PC-Gym: Benchmark Environments For Process Control Problems

RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising

Evolving Curricula with Regret-Based Environment Design

MDP Playground: An Analysis and Debug Testbed for Reinforcement Learning

SocNavGym: A Reinforcement Learning Gym for Social Navigation

MDP environments for the OpenAI Gym

CaiRL: A High-Performance Reinforcement Learning Environment Toolkit

EduQate: Generating Adaptive Curricula through RMABs in Education Settings

Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAX

Craftium: An Extensible Framework for Creating Reinforcement Learning Environments