Abstract:_ This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 206537, “Development of Deep Transformer-Based Models for Long-Term Prediction of Transient Production of Oil Wells,” by Ildar R. Abdrakhmanov, Evgenii A. Kanin, SPE, and Sergei A. Boronin, Skolkovo Institute of Science and Technology, et al. The paper has not been peer reviewed. _ The authors of the complete paper propose a novel approach to data-driven modeling of transient production of oil wells, applying transformer-based neural networks trained on multivariate time series composed of various parameters of oil wells measured during exploitation. By tuning machine-learning models for a single well (ignoring the effect of neighboring wells) on open-source field data sets, the authors of the paper demonstrate that the transformer-based method outperforms recurrent neural networks (RNNs) with long- and short-term memory (LSTM) and gated recurrent-unit cells in the forecasting of bottomhole pressure dynamics. Introduction The authors apply a novel deep-learning algorithm called a transformer to build surrogate models for simulations of well performance. Transformer architecture initially was developed for natural-language processing problems. However, in recent years, researchers have adapted transformers for time-series forecasting. In contrast with RNNs, the transformer does not process data sequentially. Instead, it handles the entire sequence by a multihead self-attention mechanism. The transformer is much more computationally efficient than an RNN because its training can be performed using graphics processing units. Transformer networks allow for application of the transfer-learning technique. First, the model is trained to predict the target parameter (e.g., bottomhole pressure) using the production data from a certain well. Next, the tuned weights are used to initialize the training procedure on the data from a target well. In other words, the fine-tuning is performed based on the pretrained model. The described technique allows the model to transfer knowledge from one time-series forecasting problem to another, leading to acceleration of the training process and improvement of model-prediction capability. The recurrent networks are not well-suited for transferring learning because the hidden state of recurrent cells is not transferred commonly. Problem Formulation The authors consider a transient production of an oil well with arbitrary completion. In most cases, various parameters of the well are measured during its exploitation, including bottomhole and wellhead pressure and temperature, flow rates for each phase, choke size, and parameters of electric centrifugal pumps (if available). In this case, the authors propose an alternative approach to find the dependence between bottomhole pressure and flow rate that is based on deep-learning algorithms trained on available field data and that allows for obtaining quick predictions of bottomhole pressure or flow rates. The data-driven coupled model of a well and reservoir can be used to plan and optimize the production process. Also, the developed model can be used to estimate well and formation properties using a constant flow-rate response of a well.

Tree-Based Ensembles for Predicting the Bottomhole Pressure of Oil and Gas Well Flows

Transformer-Based Models Aid Prediction of Transient Production of Oil Wells

Accurate artificial intelligence-based methods in predicting bottom-hole pressure in multiphase flow wells, a comparison approach

Flowing Bottomhole Pressure during Gas Lift in Unconventional Oil Wells

Enhancing pressure gradient prediction in multi-phase flow through diverse well geometries of North American shale gas fields using deep learning

A Model Based on the Random Forest Algorithm That Predicts the Total Oil–Water Two-Phase Flow Rate in Horizontal Shale Oil Wells

Prediction of Single-Well Production Rate after Hydraulic Fracturing in Unconventional Gas Reservoirs Based on Ensemble Learning Model

Development of Deep Transformer-Based Models for Long-Term Prediction of Transient Production of Oil Wells

Application of an enhanced decision tree learning approach for prediction of petroleum production

Downhole Pressure Prediction for Deep Water Gas Reservoirs Using Physics-Based and Machine Learning Models

Artificial Neural Network Prediction of Wellbore Stability in Offshore Shallow Formations

A Symbolic Tree Model for Oil and Gas Production Prediction Using Time-Series Production Data.

Deep Learning and Data Assimilation for Real-Time Production Prediction in Natural Gas Wells

A Physics-Constrained Data-Driven Workflow for Predicting Coalbed Methane Well Production Using A Combined Gated Recurrent Unit and Multi-Layer Perception Neural Network Model

Predicting production-rate using wellhead pressure for shale gas well based on Temporal Convolutional Network

A Data Driven Method to Predict and Restore Missing Well Head Flow Pressure

Optimization of the Pressure Drop Prediction Model of Wellbore Multiphase Flow Based on Simultaneous Perturbation Stochastic Approximation

Bottomhole Pressure Prediction of Carbonate Reservoirs Using XGBoost

Fully transient coupled prediction model of wellbore temperature and pressure for multi-phase flow during underbalanced drilling

Development of Machine Learning-Based Production Forecasting for Offshore Gas Fields Using a Dynamic Material Balance Equation

Data-Driven Discovery of Physics for Reservoir Surveillance