Abstract:The organic Rankine cycle (ORC) is a promising technology for engine waste heat recovery. During real-world operation, the engine working condition varies frequently to satisfy the power demand; thus, the transient nature of engine waste heat presents significant control challenges for the ORC. To control the superheat of the ORC precisely under a transient heat source, several optimal control methods have been used such as model predictive control and dynamic programing. However, most of them depend strongly on the accurate prediction of future disturbances. Deep reinforcement learning (DRL) is an artificial-intelligence algorithm that can overcome the aforementioned disadvantage, but the potential of DRL in control of thermodynamic systems has not yet been investigated. Thus, this paper proposes two DRL-based control methods for controlling the superheat of ORC under a transient heat source. One directly uses the DRL agent to learn the control strategy (DRL control), and the other uses the DRL agent to optimize the parameters of the proportional-integral-derivative (PID) controller (DRL-based PID control). Additionally, a switching mechanism between different DRL controllers is proposed for improving the training efficiency and enlarging the operation range of the controller. The results of this study indicate that the DRL agent can satisfactorily perform the control task and optimize the traditional controller under the trained and untrained transient heat source. Specifically, the DRL control can track the reference superheat with an average error of only 0.19 K, whereas that of the traditional PID control is 2.16 K. Furthermore, the proposed switching DRL control exhibits excellent tracking performance with an average error of only 0.21 K and robustness over a wide range of operation conditions. The successful application of DRL demonstrates its considerable potential for the control of thermodynamic systems, providing a useful reference and motivation for the application to other thermodynamic systems.

Close d-loop force d heat convection control using deep reinforcement learning

Closed-loop forced heat convection control using deep reinforcement learning

Control policy transfer of deep reinforcement learning based intelligent forced heat convection control

A Reinforcement Learning-Based Temperature Control of Fluidized Bed Reactor in Gas-Phase Polyethylene Process

Deep Reinforcement Learning for the Heat Transfer Control of Pulsating Impinging Jets

Performance analysis of reinforcement learning algorithms on intelligent closed-loop control on fluid flow and convective heat transfer

Deep Reinforcement Learning-Pid Based Supervisor Control Method for Indirect-Contact Heat Transfer Processes in Energy Systems

Control of Superheat of Organic Rankine Cycle under Transient Heat Source Based on Deep Reinforcement Learning

Controlling Rayleigh–Bénard convection via reinforcement learning

Deep Reinforcement Learning: A New Beacon for Intelligent Active Flow Control

A dual grid-based deep reinforcement learning and computational fluid dynamics method for flow control and its application to nucleate pool boiling

Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need

Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensing

DRLinFluids---An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM

Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows

Accelerating and improving deep reinforcement learning-based active flow control: Transfer training of policy network

Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning

Active flow control with rotating cylinders by an artificial neural network trained by deep reinforcement learning

Deep reinforcement learning for turbulent drag reduction in channel flows

Control of quasi-equilibrium state of annular flow through reinforcement learning

Deep reinforcement transfer learning of active control for bluff body flows at high Reynolds number