Abstract:_ This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 206537, “Development of Deep Transformer-Based Models for Long-Term Prediction of Transient Production of Oil Wells,” by Ildar R. Abdrakhmanov, Evgenii A. Kanin, SPE, and Sergei A. Boronin, Skolkovo Institute of Science and Technology, et al. The paper has not been peer reviewed. _ The authors of the complete paper propose a novel approach to data-driven modeling of transient production of oil wells, applying transformer-based neural networks trained on multivariate time series composed of various parameters of oil wells measured during exploitation. By tuning machine-learning models for a single well (ignoring the effect of neighboring wells) on open-source field data sets, the authors of the paper demonstrate that the transformer-based method outperforms recurrent neural networks (RNNs) with long- and short-term memory (LSTM) and gated recurrent-unit cells in the forecasting of bottomhole pressure dynamics. Introduction The authors apply a novel deep-learning algorithm called a transformer to build surrogate models for simulations of well performance. Transformer architecture initially was developed for natural-language processing problems. However, in recent years, researchers have adapted transformers for time-series forecasting. In contrast with RNNs, the transformer does not process data sequentially. Instead, it handles the entire sequence by a multihead self-attention mechanism. The transformer is much more computationally efficient than an RNN because its training can be performed using graphics processing units. Transformer networks allow for application of the transfer-learning technique. First, the model is trained to predict the target parameter (e.g., bottomhole pressure) using the production data from a certain well. Next, the tuned weights are used to initialize the training procedure on the data from a target well. In other words, the fine-tuning is performed based on the pretrained model. The described technique allows the model to transfer knowledge from one time-series forecasting problem to another, leading to acceleration of the training process and improvement of model-prediction capability. The recurrent networks are not well-suited for transferring learning because the hidden state of recurrent cells is not transferred commonly. Problem Formulation The authors consider a transient production of an oil well with arbitrary completion. In most cases, various parameters of the well are measured during its exploitation, including bottomhole and wellhead pressure and temperature, flow rates for each phase, choke size, and parameters of electric centrifugal pumps (if available). In this case, the authors propose an alternative approach to find the dependence between bottomhole pressure and flow rate that is based on deep-learning algorithms trained on available field data and that allows for obtaining quick predictions of bottomhole pressure or flow rates. The data-driven coupled model of a well and reservoir can be used to plan and optimize the production process. Also, the developed model can be used to estimate well and formation properties using a constant flow-rate response of a well.

Real‐time prediction of horizontal drilling pressure based on convolutional Transformer

Real-Time Prediction of TBM Driving Parameters Using Geological and Operation Data<italic/>

Prediction of Urban Groundwater Level Based on CNN-Transformer

Real-Time Prediction of TBM Driving Parameters Using In Situ Geological and Operation Data

Deep Learning Method for Improving Rate of Penetration Prediction in Drilling

Transformer-Based Models Aid Prediction of Transient Production of Oil Wells

Development of Deep Transformer-Based Models for Long-Term Prediction of Transient Production of Oil Wells

Drilling Rate of Penetration Prediction Based on CBT-LSTM Neural Network

Multi-Task Learning Network-Based Prediction of Hydraulic Fracturing Effects in Horizontal Wells Within the Ordos Yanchang Formation Tight Reservoir

Prediction of Leakage Pressure during a Drilling Process Based on SSA-LSTM

Predicting production-rate using wellhead pressure for shale gas well based on Temporal Convolutional Network

An Intelligent Prediction Method of Horizontal Well ROP Based on Di-GRU

Real-Time Prediction of Wellbore Trajectory with a Dual-Input GRU(Di-GRU) Model

A New Model for Predicting Surface Pump Pressure of Drilling Rig Using Artificial Neural Network

Real-time prediction of TBM penetration rates using a transformer-based ensemble deep learning model

Multi-source information fusion-based dynamic model for online prediction of rate of penetration (ROP) in drilling process

An Efficient Approach for Real-Time Prediction of Rate of Penetration in Offshore Drilling

Transient pressure prediction in large-scale underground natural gas storage: A deep learning approach and case study

Study on Rock Mechanics Parameter Prediction Method Based on DTW Similarity and Machine-Learning Algorithms

Porosity and Permeability Prediction Using a Transformer and Periodic Long Short-Term Network

Predicting Pore-Water Pressure in Front of a TBM Using a Deep Learning Approach