Abstract:Abstract Steam injection is a popular technique to enhance oil recovery in mature oil fields. However, the conventional approach of using a constant steam rate over an extended period can lead to sub-optimal performance due to the complex nature of the problem and reservoir heterogeneity. To address this issue, the Markov decision process can be employed to formulate the problem for reinforcement learning (RL) applications. The RL agent is trained to optimize the steam injection rate by interacting with a reservoir simulation model and receives rewards for each action. The agent’s policy and value functions are updated through continuous interaction with the environment until convergence is achieved, leading to a more efficient steam injection strategy for enhancing oil recovery. In this study, an actor-critic RL architecture was employed to train the agent to find the optimal strategy (i.e., policy). The environment was represented by a reservoir simulation model, and the agent’s actions were based on the observed state. The policy function gave a probability distribution of the actions that the agent could take, while the value function determined the expected yield for an agent starting from a given state. The agent interacted with the environment for several episodes until convergence was achieved. The improvement in net present value (NPV) achieved by the agent was a significant indication of the effectiveness of the RL-based approach. The NPV reflects the economic benefits of the optimized steam injection strategy. The agent was able to achieve this improvement by finding the optimal policies. One of the key advantages of the optimal policy was the decrease in total field heat losses. This is a critical factor in the efficiency of the steam injection process. Heat loss can reduce the efficiency of the process and lead to lower oil recovery rates. By minimizing heat loss, the agent was able to optimize the steam injection process and increase oil recovery rates. The optimal policy had four regions characterized by slight changes in a stable injection rate to increase the average reservoir pressure, increasing the injection rate to a maximum value, steeply decreasing the injection rate, and slightly changing the injection rate to maintain the average reservoir temperature. These regions reflect the different phases of the steam injection process and demonstrate the complexity of the problem. Overall, the results of this study demonstrate the effectiveness of RL in optimizing steam injection in mature oil fields. The use of RL can help address the complexity of the problem and improve the efficiency of the oil recovery process. This study provides a framework for future research in this area and highlights the potential of RL for addressing other complex problems in the energy industry.

Assessment of reinforcement learning algorithms for nuclear power plant fuel optimization

Assessment of Reinforcement Learning Algorithms for Nuclear Power Plant Fuel Optimization

Physics-informed Reinforcement Learning optimization of PWR core loading pattern

Surpassing legacy approaches to PWR core reload optimization with single-objective Reinforcement learning

Optimizing the Fixed Number Detector Placement for the Nuclear Reactor Core Using Reinforcement Learning

Multi-Objective Reinforcement Learning-based Approach for Pressurized Water Reactor Optimization

Reactor Optimization Benchmark by Reinforcement Learning

Integrating the Deep Learning and Multi-Objective Genetic Algorithm to the Reloading Pattern Optimization of HPR1000 Reactor Core

Possibilities of reinforcement learning for nuclear power plants: Evidence on current applications and beyond

A Safe Reinforcement Learning Algorithm for Supervisory Control of Power Plants

Multistep Criticality Search and Power Shaping in Microreactors with Reinforcement Learning

Design Optimization of Nuclear Fusion Reactor through Deep Reinforcement Learning

Multi-objective optimization of thermal power and outlet steam temperature for a nuclear steam supply system with deep reinforcement learning

Reinforcement learning-trained optimisers and Bayesian optimisation for online particle accelerator tuning

Injection Optimization at Particle Accelerators via Reinforcement Learning: From Simulation to Real-World Application

A Novel Method for Controlling Crud Deposition in Nuclear Reactors Using Optimization Algorithms and Deep Neural Network Based Surrogate Models

Optimizing Load Scheduling in Power Grids Using Reinforcement Learning and Markov Decision Processes

Integrating an Ensemble Reward System into an Off-Policy Reinforcement Learning Algorithm for the Economic Dispatch of Small Modular Reactor-Based Energy Systems

Actor-critic reinforcement learning leads decision-making in energy systems optimization—steam injection optimization

Optimal dispatch of an energy hub with compressed air energy storage: A safe reinforcement learning approach