Abstract:As the demand for precise predictions grows across various industries due to advancements in sensor technology and computer hardware, multi-feature time series prediction shows significant promise in fields such as information fusion, finance, energy, and meteorology. However, traditional machine learning methods often struggle to forecast future events given the increasing complexity of the data. To address this challenge, the paper introduces an innovative approach that combines an improved k -means clustering with a multi-head self-attention mechanism. This method utilizes long and short-term memory (LSTM) neural networks to filter and identify the most effective feature subset for prediction. In the enhanced k -means clustering algorithm, a novel similarity formula named Feature Vector Similarity (FVS) and a method for automatically determining the number of clustering centers are proposed. This advancement improves the rationality of cluster center selection and enhances overall clustering performance. The multi-head self-attention mechanism calculates the clustering centers and attention weights of objects within the cluster partitions, optimizing feature selection and enhancing computational efficiency. The fusion of k -means clustering, the multi-head self-attention mechanism, and LSTM networks results in a new feature selection method, referred to as KMAL. To further refine the prediction process, we integrate KMAL with LSTM, known for its strong performance in predicting long-term time series, to develop a novel prediction model: KMAL-LSTM. In the subsequent comparative experiments, the prediction performance of the models is assessed using mean absolute error (MAE), mean bias error (MBE), and root mean square error (RMSE). The proposed KMAL-LSTM model consistently exhibits superior validity, stability, and performance when compared to seven other prediction models across six distinct datasets.

A Hybrid Method with Adaptive Sub-Series Clustering and Attention-Based Stacked Residual LSTMs for Multivariate Time Series Forecasting

Foreformer: an Enhanced Transformer-Based Framework for Multivariate Time Series Forecasting

Effective LSTMs with Seasonal-Trend Decomposition and Adaptive Learning and Niching-Based Backtracking Search Algorithm for Time Series Forecasting

Adapt to Small-Scale and Long-Term Time Series Forecasting with Enhanced Multidimensional Correlation

TCLN: A Transformer-based Conv-LSTM Network for Multivariate Time Series Forecasting

Generalizable Memory-driven Transformer for Multivariate Long Sequence Time-series Forecasting

An Ensemble Model Based on Adaptive Noise Reducer and Over-Fitting Prevention LSTM for Multivariate Time Series Forecasting.

A Novel Encoder-Decoder Model for Multivariate Time Series Forecasting

Temporal Self-Attention-based Conv-LSTM Network for Multivariate Time Series Prediction

Fusion k-means clustering and multi-head self-attention mechanism for a multivariate time prediction model with feature selection

AFSTGCN: Prediction for multivariate time series using an adaptive fused spatial-temporal graph convolutional network

Parallel Extraction of Long-term Trends and Short-term Fluctuation Framework for Multivariate Time Series Forecasting

Long Time Series Deep Forecasting with Multiscale Feature Extraction and Seq2seq Attention Mechanism

A dual‐stage attention‐based Conv‐LSTM network for spatio‐temporal correlation and multivariate time series prediction

TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment

A dual-stage attention-based Bi-LSTM network for multivariate time series prediction

Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modeling

FMamba: Mamba based on Fast-attention for Multivariate Time-series Forecasting

Adaptive Multi-Scale Decomposition Framework for Time Series Forecasting

Sequence Attention for Multivariate Time Series Forecasting

Is Single Enough? A Joint Spatiotemporal Feature Learning Framework for Multivariate Time Series Prediction