Abstract:Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, Ahead of Print. A deep reinforcement learning application is investigated to control the emissions of a compression ignition diesel engine. The main purpose of this study is to reduce the engine-out nitrogen oxide [math] emissions and to minimize fuel consumption while tracking a reference engine load. First, a physics-based engine simulation model is developed in GT-Power and calibrated using experimental data. Using this model and a GT-Power/Simulink co-simulation, a deep deterministic policy gradient is developed. To reduce the risk of an unwanted output, a safety filter is added to the deep reinforcement learning. Based on the simulation results, this filter has no effect on the final trained deep reinforcement learning; however, during the training process, it is crucial to enforce constraints on the controller output. The developed safe reinforcement learning is then compared with an iterative learning controller and a deep neural network–based nonlinear model predictive controller. This comparison shows that the safe reinforcement learning is capable of accurately tracking an arbitrary reference input while the iterative learning controller is limited to a repetitive reference. The comparison between the nonlinear model predictive control and reinforcement learning indicates that for this case reinforcement learning is able to learn the optimal control output directly from the experiment without the need for a model. However, to enforce output constraint for safe learning reinforcement learning, a simple model of system is required. In this work, reinforcement learning was able to reduce [math] emissions more than the nonlinear model predictive control; however, it suffered from slightly higher error in load tracking and a higher fuel consumption.

A Reinforcement Learning Approach for Transient Control of Liquid Rocket Engines

An MPC Approach to Transient Control of Liquid-Propellant Rocket Engines

Accelerating Reinforcement Learning with Local Data Enhancement for Process Control

Intelligent Control Strategy for Transient Response of a Variable Geometry Turbocharger System Based on Deep Reinforcement Learning

Turbopump Parametric Modelling and Reliability Assessment for Reusable Rocket Engine Applications

A reinforcement learning approach for adaptive tracking control of a reusable rocket model in a landing scenario

A Study of Aero-Engine Control Method Based on Deep Reinforcement Learning.

Real-time accommodation of actuator faults on a reusable rocket engine

Realizing Stabilized Landing for Computation-Limited Reusable Rockets: A Quantum Reinforcement Learning Approach

Safe deep reinforcement learning in diesel engine emission control

Thrust Control Method and Technology of Variable-Thrust Liquid Engine for Reusable Launch Rocket

Rocket Landing Control with Random Annealing Jump Start Reinforcement Learning

Transfer of Reinforcement Learning-Based Controllers from Model- to Hardware-in-the-Loop

Deep reinforcement learning based active surge control for aeroengine compressors

Reinforcement Learning Control of Hypersonic Vehicles and Performance Evaluations

Machine Learning Methods for the Design and Operation of Liquid Rocket Engines -- Research Activities at the DLR Institute of Space Propulsion

Human operator decision support for highly transient industrial processes: a reinforcement learning approach

Composite Observer-Based Optimal Attitude-Tracking Control With Reinforcement Learning for Hypersonic Vehicles

Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows

Multistep Criticality Search and Power Shaping in Microreactors with Reinforcement Learning

Optimizing Solid Rocket Missile Trajectories: A Hybrid Approach Using an Evolutionary Algorithm and Machine Learning