Abstract:Electric motors are used in many applications, and their efficiency is strongly dependent on their control. Among others, linear feedback approaches or model predictive control methods are well known in the scientific literature and industrial practice. A novel approach is to use reinforcement learning (RL) to have an agent learn electric drive control from scratch merely by interacting with a suitable control environment. RL achieved remarkable results with superhuman performance in many games (e.g., Atari classics or Go) and also becomes more popular in control tasks, such as cart-pole or swinging pendulum benchmarks. In this work, the open-source Python package gym-electric-motor (GEM) is developed for ease of training of RL-agents for electric motor control. Furthermore, this package can be used to compare the trained agents with other state-of-the-art control approaches. It is based on the OpenAI Gym framework that provides a widely used interface for the evaluation of RL-agents. The package covers different dc and three-phase motor variants, as well as different power electronic converters and mechanical load models. Due to the modular setup of the proposed toolbox, additional motor, load, and power electronic devices can be easily extended in the future. Furthermore, different secondary effects, such as converter interlocking time or noise, are considered. An intelligent controller example based on the deep deterministic policy gradient algorithm that controls a series dc motor is presented and compared to a cascaded proportional–integral controller as a baseline for future research. Here, safety requirements are particularly highlighted as an important constraint for data-driven control algorithms applied to electric energy systems. Fellow researchers are encouraged to use the GEM framework in their RL investigations or contribute to the functional scope (e.g., further motor types) of the package.

ModelicaGym: Applying Reinforcement Learning to Modelica Models

Towards a Reinforcement Learning Environment Toolbox for Intelligent Electric Motor Control

OR-Gym: A Reinforcement Learning Library for Operations Research Problems

Toward a Reinforcement Learning Environment Toolbox for Intelligent Electric Motor Control

Gymnasium: A Standard Interface for Reinforcement Learning Environments

Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms

EduGym: An Environment and Notebook Suite for Reinforcement Learning Education

Gym-ANM: Open-source software to leverage reinforcement learning for power system management in research and education

PC-Gym: Benchmark Environments For Process Control Problems

Model-Based Policy Search Using Monte Carlo Gradient Estimation with Real Systems Application

Reinforcement Learning for Thermostatically Controlled Loads Control using Modelica and Python

gym-gazebo2, a toolkit for reinforcement learning using ROS 2 and Gazebo

safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Robotics

Gym-preCICE: Reinforcement Learning Environments for Active Flow Control

COOL-MC: A Comprehensive Tool for Reinforcement Learning and Model Checking

SDGym: Low-Code Reinforcement Learning Environments using System Dynamics Models

Flexible development and evaluation of machine‐learning‐supported optimal control and estimation methods via HILO‐MPC

Extending the OpenAI Gym for robotics: a toolkit for reinforcement learning using ROS and Gazebo

LapGym -- An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery

A Model-Based Reinforcement Learning Approach for PID Design

Model Learning for Two-Wheeled Robot Self-Balance Control