Reinforcement Learning Versus Model Predictive Control on Greenhouse Climate Control

Bernardo Morcego,Wenjie Yin,Sjoerd Boersma,Eldert van Henten,Vicenç Puig,Congcong Sun

2023-03-11

Abstract:Greenhouse is an important protected horticulture system for feeding the world with enough fresh food. However, to maintain an ideal growing climate in a greenhouse requires resources and operational costs. In order to achieve economical and sustainable crop growth, efficient climate control of greenhouse production becomes essential. Model Predictive Control (MPC) is the most commonly used approach in the scientific literature for greenhouse climate control. However, with the developments of sensing and computing techniques, reinforcement learning (RL) is getting increasing attention recently. With each control method having its own way to state the control problem, define control goals, and seek for optimal control actions, MPC and RL are representatives of model-based and learning-based control approaches, respectively. Although researchers have applied certain forms of MPC and RL to control the greenhouse climate, very few effort has been allocated to analyze connections, differences, pros and cons between MPC and RL either from a mathematical or performance perspective. Therefore, this paper will 1) propose MPC and RL approaches for greenhouse climate control in an unified framework; 2) analyze connections and differences between MPC and RL from a mathematical perspective; 3) compare performance of MPC and RL in a simulation study and afterwards present and interpret comparative results into insights for the application of the different control approaches in different scenarios.

Optimization and Control

What problem does this paper attempt to address?

This paper attempts to solve two main problems in greenhouse climate control:

1. **Improve the economic benefits and sustainability of greenhouse crop production**: In order to produce enough fresh food on limited land, greenhouses need to maintain an ideal growth environment. However, this requires a large amount of resources and operating costs. Therefore, how to achieve economic and sustainable crop growth through efficient climate control is a key issue.

2. **Compare the applications of model predictive control (MPC) and reinforcement learning (RL) in greenhouse climate control**: Although MPC is the most commonly used method in greenhouse climate control at present, with the development of sensing and computing technologies, RL has gradually attracted attention. However, few studies have analyzed the connections, differences, and respective advantages and disadvantages between these two methods from a mathematical or performance perspective.

### Specific Objectives

The specific objectives of the paper include:

1. **Propose MPC and RL methods under a unified framework**: Apply MPC and RL to greenhouse climate control and describe them under a unified framework.

2. **Analyze the connections and differences between MPC and RL from a mathematical perspective**: Explore in detail the differences between the two methods in formulating control problems, defining control objectives, and finding optimal control actions.

3. **Compare the performance of MPC and RL through simulation studies**: Simulate MPC and RL under the same conditions and analyze their performance differences, so as to provide guidance for applications in different scenarios.

### Method Overview

- **MPC method**: Based on a nonlinear model, minimize a constrained cost function by optimizing the future control input sequence. MPC adopts the receding horizon principle, updates the model at each time step and applies the first value of the optimal control input.

- **RL method**: Develop an RL controller using the deep deterministic policy gradient (DDPG) algorithm. DDPG is an algorithm based on the actor - critic structure and can show good performance in continuous state - action spaces. RL guides the learning process through a reward function and a discount factor.

### Summary

This paper aims to provide theoretical basis and practical guidance for the future design of greenhouse control systems by comparing the applications of MPC and RL in greenhouse climate control, in order to achieve more efficient and more economical crop production.