Abstract:Controlling greenhouse crop production systems is a complex task due to uncertain and non-linear dynamics between crops, indoor and outdoor climate, and economics. The declining number of skilled growers necessitates the development of autonomous greenhouse control systems. Reinforcement Learning (RL) is a promising approach that can learn a control policy to automate greenhouse management. RL optimises a control policy through interactions with a model of the greenhouse while guided by an economic-based reward function. However, its application to real-world systems is limited due to discrepancies between models and real-world dynamics. Moreover, RL controllers may struggle to maintain state constraints while optimising the primary objective, especially when models inadequately capture the adverse effects of constraint violations on crop growth. Also, the generalisation to novel states, for example, due to unseen weather trajectories, is underexplored in RL-based greenhouse control. This work addresses these challenges through three key contributions. First, we present GreenLight-Gym, the first open-source environment designed for training and evaluating RL algorithms on the state-of-the-art greenhouse model GreenLight. GreenLight-Gym enables the community to benchmark RL-based control methodologies. Second, we compare two reward-shaping approaches, using either a multiplicative or additive penalty, to enforce state boundaries. The additive penalty achieves more stable training while better adhering to state constraints, while the multiplicative penalty yields marginally higher profits. Finally, we evaluate RL performance on a disjoint training and testing weather dataset, demonstrating improved generalisation to unseen conditions. Our environment and experiment scripts are open-sourced, facilitating innovative research on learning-based greenhouse control.

Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms

safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Robotics

PDE Control Gym: A Benchmark for Data-Driven Boundary Control of Partial Differential Equations

PC-Gym: Benchmark Environments For Process Control Problems

Gymnasium: A Standard Interface for Reinforcement Learning Environments

RobocupGym: A challenging continuous control benchmark in Robocup

EduGym: An Environment and Notebook Suite for Reinforcement Learning Education

Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark

Gym-preCICE: Reinforcement Learning Environments for Active Flow Control

Double A3C: Deep Reinforcement Learning on OpenAI Gym Games

SDGym: Low-Code Reinforcement Learning Environments using System Dynamics Models

Gym-ANM: Open-source software to leverage reinforcement learning for power system management in research and education

OffWorld Gym: open-access physical robotics environment for real-world reinforcement learning benchmark and research

Applications in CityLearn Gym Environment for Multi-Objective Control Benchmarking in Grid-Interactive Buildings and Districts

MDP Playground: An Analysis and Debug Testbed for Reinforcement Learning

LapGym -- An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery

Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control

Towards a Reinforcement Learning Environment Toolbox for Intelligent Electric Motor Control

GreenLight-Gym: A Reinforcement Learning Benchmark Environment for Greenhouse Crop Production Control

An Optical Control Environment for Benchmarking Reinforcement Learning Algorithms

ChemGymRL: An Interactive Framework for Reinforcement Learning for Digital Chemistry