Abstract:The emergence of Long Short-Term Memory (LSTM) solves the problems of vanishing gradient and exploding gradient in traditional Recurrent Neural Networks (RNN). LSTM, as a new type of RNN, has been widely used in various fields, such as text prediction, Wind Speed Forecast, depression prediction by EEG signals, etc. The results show that improving the efficiency of LSTM can help to improve the efficiency in other application areas. In this paper, we proposed an advanced LSTM algorithm, the Extreme Long Short-Term Memory (E-LSTM), which adds the inverse matrix part of Extreme Learning Machine (ELM) as a new "gate" into the structure of LSTM. This "gate" preprocess a portion of the data and involves the processed data in the cell update of the LSTM to obtain more accurate data with fewer training rounds, thus reducing the overall training time. In this research, the E-LSTM model is used for the text prediction task. Experimental results showed that the E-LSTM sometimes takes longer to perform a single training round, but when tested on a small data set, the new E-LSTM requires only 2 epochs to obtain the results of the 7th epoch traditional LSTM. Therefore, the E-LSTM retains the high accuracy of the traditional LSTM, whilst also improving the training speed and the overall efficiency of the LSTM.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the problems of training efficiency and accuracy in time - series prediction tasks of the traditional Long - Short - Term Memory network (LSTM). Specifically, the paper proposes an improved LSTM model - Extreme Long - Short - Term Memory (E - LSTM), optimizing the structure and performance of LSTM by introducing the idea of Extreme Learning Machine (ELM). #### Main problems: 1. **Vanishing and exploding gradient problems**: Traditional Recurrent Neural Networks (RNN) are prone to vanishing and exploding gradient problems when dealing with long - time - series data, which makes it difficult for the model to converge and leads to poor performance. 2. **Low training efficiency**: Although LSTM has solved the gradient problem to a certain extent, in some tasks, there are still problems such as overly long training time and the need for a large number of iterations to achieve high accuracy. 3. **Improving the accuracy of time - series prediction**: How to maintain or even improve the prediction accuracy of the model while reducing the number of training rounds is an important research direction. #### Solutions: To address the above problems, the paper proposes the E - LSTM model, and its main innovations include: - **Introducing the "gate" mechanism of ELM**: Taking the inverse matrix part of ELM as a new "gate" and adding it to the LSTM structure. This new "gate" is responsible for pre - processing part of the data and making its results participate in the cell update process of LSTM. - **Reducing the number of training rounds**: By combining the fast - learning characteristics of ELM, E - LSTM can obtain similar or even higher accuracy compared with traditional LSTM within fewer training rounds, thus significantly reducing the overall training time. - **Improving training efficiency**: Experimental results show that although the time of a single training round of E - LSTM is slightly longer than that of traditional LSTM, the overall training time is greatly reduced, and it can achieve higher accuracy within the same time. ### Summary: This paper improves LSTM by introducing the idea of ELM and proposes a new E - LSTM model, aiming to solve the problems of low training efficiency and insufficient accuracy of traditional LSTM in time - series prediction tasks. Experimental results show that E - LSTM not only improves the training speed but also shows better prediction performance on multiple datasets.

Extreme-Long-short Term Memory for Time-series Prediction

Deep Learning with Long Short-Term Memory for Time Series Prediction

EA-LSTM: Evolutionary attention-based LSTM for time series prediction

A novel error-output recurrent two-layer extreme learning machine for multi-step time series prediction

Fuzzy inference-based LSTM for long-term time series prediction

Sensitive time series prediction using extreme learning machine

Time Series Prediction Method Based on Variant LSTM Recurrent Neural Network

Time Series Prediction Based on LSTM-Attention-LSTM Model

NEWLSTM: an Optimized Long Short-Term Memory Language Model for Sequence Prediction.

SALSTM: Segmented Self-Attention Long Short-Term Memory for Long-Term Forecasting

Prediction Based on Online Extreme Learning Machine in WWTP Application

ELM-KL-LSTM: a robust and general incremental learning method for efficient classification of time series data

Unlocking the Power of LSTM for Long Term Time Series Forecasting

DAFA-BiLSTM: Deep Autoregression Feature Augmented Bidirectional LSTM network for time series prediction

ELSTM: An improved long short‐term memory network language model for sequence learning

Multi-step Ahead Time Series Forecasting for Different Data Patterns Based on LSTM Recurrent Neural Network.

NOA-LSTM: An Efficient LSTM cell architecture for Time Series forecasting

xLSTMTime : Long-term Time Series Forecasting With xLSTM

LSTM Model for Prediction of Heart Failure in Big Data

Short-Term Prediction in Vessel Heave Motion Based on Improved LSTM Model

Advanced LSTM: A Study about Better Time Dependency Modeling in Emotion Recognition